return2ozma@lemmy.world to Technology@lemmy.worldEnglish · 5 days agoApple has new Mac app coming to make running local AI models easy9to5mac.comexternal-linkmessage-square26linkfedilinkarrow-up1113arrow-down122
arrow-up191arrow-down1external-linkApple has new Mac app coming to make running local AI models easy9to5mac.comreturn2ozma@lemmy.world to Technology@lemmy.worldEnglish · 5 days agomessage-square26linkfedilink
minus-squareCardiac@lemmy.mllinkfedilinkEnglisharrow-up7arrow-down2·5 days agoDynamic pooling could make this more accessible to lower power macs. Are there other handlers that can bundle computers into a local Ai instance in a similar manner?
minus-squaresetsubyou@lemmy.worldlinkfedilinkEnglisharrow-up1·5 days agoI guess vLLM supports clusters so you could just use that? The setup is more difficult than just plugging in a cable though.
Dynamic pooling could make this more accessible to lower power macs.
Are there other handlers that can bundle computers into a local Ai instance in a similar manner?
I guess vLLM supports clusters so you could just use that? The setup is more difficult than just plugging in a cable though.