3 ms·
I'm pretty excited about the potential of on-package memory to enable LLM's and other AI stuff to run on local hardware with higher performance than using DDR D
by faeriechangling 4y ago
I'm pretty excited about the potential of on-package memory to enable LLM's and other AI stuff to run on local hardware with higher performance than using DDR DIMMs and lower cost than shelling out 80 grand for a pair of H100s.
When the arguments for DIMMs are "field upgrades" which is a problem you can avoid by buying more memory to begin with (Also in practice few users actually upgrade memory), and "price segmentation" which isn't really a technical consideration, I see the writing on the wall. At least in the consumer space.
- rasz 4y agoYou are excited about swapping out somehow making for better performance? Even biggest M2 Max cant fit bigger models. I dont know anyone with older macbook who didnt max out ram as soon as it became financially sensible.
- alfor 4y agoI agree with you. I think Apple is trailblazing again. Most laptop will go in that direction. Tighter integration will lead to a better performance/cost and better efficiency also. AMD 3D cache is also going in that direction integrating ram/cache on top of cpu with high density of interconnect.