3 ms·
Anyone on HN running models on their own local machines, like smaller Llama models or such? Or something else?
by sgt 2y ago
Anyone on HN running models on their own local machines, like smaller Llama models or such? Or something else?
- karpatic 2y agoFor sure dude! Top comment thread is all about using ollama and other ways to get that done.
- grahamj 2y agoDoesn’t everyone? X) it’s super easy now with ollama + openwebui or an all in 1 like mlstudio
- sgt 2y agoWas just concerned I don't have enough RAM. I have 16GB (M2 Pro). Got amazing mem bandwidth though (800GB/s)
- kristianp 2y agoM2 Pro has 200GB/s
- sgt 2y agoYou're right. M1 and M2 Pro: 200GB/s M3 Max: 300GB/s M1/M2 Max: 400GB/s M1/M2 Ultra: 800GB/s Seems to be the case. An Ultra .. wow. But 200GB is also still good, so not complaining.
- grahamj 2y agoIt's on the low side but plenty for something like this new 3b model. Anything up to 8GB and you've still got as much as a base model Air left over.