3 ms·

I can see how the apple silicon memory system can help with LLMs, but a couple points of reality check: - such amounts of memory is locked behind very expensiv

I can see how the apple silicon memory system can help with LLMs, but a couple points of reality check: - such amounts of memory is locked behind very expensive sku which even most of mac userbase will not use ( <5% in the new purchases to be very conservative ). - not too long ago apple would restrict the amount of ram in their system for their own reasoning (source: https://9to5mac.com/2016/10/28/apple-macbook-pro-16gb-ram-limitation-battery-efficiency/ https://9to5mac.com/2016/10/28/apple-macbook-pro-16gb-ram-li...) - just like mid 2010s gpus with 6-8 gb vram but with little to benefit from it, i don't see the ml accelerators/gpu in current models being capable enough to make the most of the memory available to it.

anentropic 4y ago
That's today... think of the future My first computer had 512KB RAM and 20MB was an expensive hard drive. 64GB Macbooks are currently an expensive 'Pro' novelty, they will be the vanilla of tomorrow > i don't see the ml accelerators/gpu in current models being capable enough to make the most of the memory available to it that's exactly my point (and apparently today's Neural Engine can't even take advantage of all the unified memory available) until LLaMA there was no reason to have more than this, they probably imagined it would just run a bit of face-detection and speech-to-text on the side but if they got serious and beefed it up it could be the next wave of computing IMHO