4 ms·
Apple is in such an interesting position right now. Having made the decision for whatever reason to go with a unified memory architecture, they are now almost u
by zmmmmm 1y ago
Apple is in such an interesting position right now. Having made the decision for whatever reason to go with a unified memory architecture, they are now almost unique in terms of being able to offer super power efficient laptops with very large GPU memory. If they were to just double the memory bandwidth and do nothing else they could break away as a preferred way to run especially inference but maybe even training for small LLM models. Given the bottlenecks on availability and pricing of nVidia cards we are already toying with the idea of building a small local LLM stack based on Macs for this reason. If Apple made it more compelling we could easily go from toying with it to seriously considering it.
It's disappointing to hear they are delaying this year's Macbooks because I was really hoping to see some improvements like this specifically targeted at running local LLMs.
- pyaamb 1y agowhich is why I was surprised to hear some reports that Apple was planning to abandon UMA in future chips. Can't imagine why they would do that
- makeitdouble 1y ago> they are now almost unique AMD and Intel already caught up in the unified memory design. Apple might still have some edge (do they ? idk), but it's not a rarety anymore.
- zmmmmm 1y agoThe edge is, "it actually works". Eg: I just set the device to "mps" in PyTorch, everything is using the GPU.Until nVidia gets monopoly treatment for CUDA and are forced to open their software stack, Metal is the next best thing as far as I can tell. Not to mention, I have no faith that these laptops won't thermally throttle - my fan only spins up when I'm running truly demanding workloads, the rest of the time the laptop might as well be fanless.
- pjmlp 1y agoIntegrated GPUs predate M chips, but naturally it is yet another thing Apple has "invented".