2 ms·
Incredible hardware. Love that I can also run local llms on mine. https://github.com/Aider-AI/aider/issues/4526 https://github.com/Aider-AI/aider/issues/4526
by __mharrison__ 1y ago
Incredible hardware. Love that I can also run local llms on mine. https://github.com/Aider-AI/aider/issues/4526 https://github.com/Aider-AI/aider/issues/4526
- amelius 1y agoBut are these llms worth their salt?
- teaearlgraycold 1y agoWith 128GB of memory they can have real world use cases. But they won’t be as good as SoTA hosted models.
- BoorishBears 1y agoThey're not unless you curve the grading because they're running locally. Which some people do, but I don't think the average person asking this question does (and I don't)
- bigyabai 1y agoIf you bought a fully-featured computer that supports compute shaders and didn't run local LLMs, you should be protesting in the street.
- ericmcer 1y agoCan't you run small LLMs on like... a Macbook air M1? Some models are under 1B weights, they will be almost useless but I imagine you could run them on anything from the last 10 years. But yeah if you wanna run 600B+ weights models your gonna need an insane setup to run it locally.
- jen729w 1y agoThey "run" in the most technical sense, yes. But they're unusably slow.
- zero_bias 1y agoI run qwen models on MBA M4 16 Gb and MBP M2 Max 32 Gb, MBA is able to handle models in accordance with its vram memory capacity (with external cooling), e.g. qwen3 embedding 8B (not 1B!) but inference is 4x-6x times slower than on mbp. I suspect weaker SoC Anyway, Apple SoC in M series is a huge leverage thanks to shared memory: VRAM size == RAM size so if you buy M chip with 128+ Gb memory, you’re pretty much able to run SOTA models locally, and price is significantly lower than AI GPU cards