5 ms·

I thought we were going into chiplets and tighter integrations. A bit like what Apple did with M1/M2, putting ram/flash/cpu/gpu on the same package with very s

by alfor 4y ago

I thought we were going into chiplets and tighter integrations. A bit like what Apple did with M1/M2, putting ram/flash/cpu/gpu on the same package with very short, wide and fast interconnects. Leaving only IO, specialized chips and power outside of the main package.

Latty 4y ago
For consumer devices that don't need much RAM, that makes sense, but for servers, I imagine it'll end up as more of an (optionally) tiered set-up, if you look at the Sapphire Rapids chips with HBM on, they can be run with only that, but they can also be run with it as another cache layer, or as a separate faster memory region. If you need an absolute ton of memory, having it all directly on the die becomes impractical at some point.
tjoff 4y ago
Hope not, that means you'd only get mainstream hardware. Incredibly limiting and boring.
faeriechangling 4y ago
DIMMS aren't going anywhere anytime soon even if it's plausible that they're going to get phased out of consumer tier products. There's way too many databases out there that need absurd amounts of memory to run performantly. Epyc supports 6tb per socket - try getting that much memory on-package any time soon.
rasz 4y ago
Apple marketing worked great on you. Putting ram on package was to upsell you on a higher more expensive model and prevent life cycle prolonging field upgrades. Anything below 1024bits is just cost cutting and forces market segmentation. Xbox has no problem fanning out 320bit bus in addition to pcie lanes, same for hiend GPUs. HMB Apple was alluding to but carefully not saying out loud starts at 4096bits. M1/M2 ram is just 128bit like any other normal mobile processor. M1/M2 Pro comes with 256bit bus (2 custom LPDDR5 chips) and 200 GB/s, M1/M2 Max with 512bit. POP ram makes sure nobody will be able to upgrade, hardware locked recurring revenue.
- faeriechangling 4y ago
  I'm pretty excited about the potential of on-package memory to enable LLM's and other AI stuff to run on local hardware with higher performance than using DDR DIMMs and lower cost than shelling out 80 grand for a pair of H100s. When the arguments for DIMMs are "field upgrades" which is a problem you can avoid by buying more memory to begin with (Also in practice few users actually upgrade memory), and "price segmentation" which isn't really a technical consideration, I see the writing on the wall. At least in the consumer space.
  - rasz 4y ago
    You are excited about swapping out somehow making for better performance? Even biggest M2 Max cant fit bigger models. I dont know anyone with older macbook who didnt max out ram as soon as it became financially sensible.
  - alfor 4y ago
    I agree with you. I think Apple is trailblazing again. Most laptop will go in that direction. Tighter integration will lead to a better performance/cost and better efficiency also. AMD 3D cache is also going in that direction integrating ram/cache on top of cpu with high density of interconnect.

by alfor 4y ago

Latty 4y ago

For consumer devices that don't need much RAM, that makes sense, but for servers, I imagine it'll end up as more of an (optionally) tiered set-up, if you look at the Sapphire Rapids chips with HBM on, they can be run with only that, but they can also be run with it as another cache layer, or as a separate faster memory region. If you need an absolute ton of memory, having it all directly on the die becomes impractical at some point.

tjoff 4y ago

Hope not, that means you'd only get mainstream hardware. Incredibly limiting and boring.

faeriechangling 4y ago

DIMMS aren't going anywhere anytime soon even if it's plausible that they're going to get phased out of consumer tier products. There's way too many databases out there that need absurd amounts of memory to run performantly. Epyc supports 6tb per socket - try getting that much memory on-package any time soon.

rasz 4y ago

Apple marketing worked great on you. Putting ram on package was to upsell you on a higher more expensive model and prevent life cycle prolonging field upgrades. Anything below 1024bits is just cost cutting and forces market segmentation. Xbox has no problem fanning out 320bit bus in addition to pcie lanes, same for hiend GPUs. HMB Apple was alluding to but carefully not saying out loud starts at 4096bits. M1/M2 ram is just 128bit like any other normal mobile processor. M1/M2 Pro comes with 256bit bus (2 custom LPDDR5 chips) and 200 GB/s, M1/M2 Max with 512bit. POP ram makes sure nobody will be able to upgrade, hardware locked recurring revenue.

faeriechangling 4y ago
I'm pretty excited about the potential of on-package memory to enable LLM's and other AI stuff to run on local hardware with higher performance than using DDR DIMMs and lower cost than shelling out 80 grand for a pair of H100s. When the arguments for DIMMs are "field upgrades" which is a problem you can avoid by buying more memory to begin with (Also in practice few users actually upgrade memory), and "price segmentation" which isn't really a technical consideration, I see the writing on the wall. At least in the consumer space.
- rasz 4y ago
  You are excited about swapping out somehow making for better performance? Even biggest M2 Max cant fit bigger models. I dont know anyone with older macbook who didnt max out ram as soon as it became financially sensible.
- alfor 4y ago
  I agree with you. I think Apple is trailblazing again. Most laptop will go in that direction. Tighter integration will lead to a better performance/cost and better efficiency also. AMD 3D cache is also going in that direction integrating ram/cache on top of cpu with high density of interconnect.