3 ms·

> Embedded RAM, like or not, is the future of CPUs unless you want memory performance/power efficiency to freeze near current levels Why is that?

bryanlarsen 3y ago
M2 pro/max/ultra bandwidth is 200/400/800 GBps. Ryzen dual channel bandwidth is 100GBps. EPYC 8 channel bandwidth is well under 400GBps.
- brucethemoose2 3y ago
  And more importantly, M2's energy/bit is pretty low.
- matja 3y ago
  * Shared bandwidth between CPU and GPU - as soon as you use up one, you're bottlenecking the other. * No M2 memory benchmark I've seen gets above ~250GB/s once cache effects are exceeded. * Random access latency is >~110ns which is no improvement over even mid-range DDR4 designs from 5 years ago. TL;DR: It's about cost
brucethemoose2 3y ago
Because the signal integrity across the pins and traces becomes a serious bottleneck. SODIMMs are already "past" the limit, requiring 1.35V for modest DDR5 speeds at terrible timings, hence vendors are trying to migrate to this: https://www.anandtech.com/show/21069/modular-lpddr-becomes-a-reality-samsung-introduces-lpcamm-memory-modules https://www.anandtech.com/show/21069/modular-lpddr-becomes-a... I suspect regular desktop/server DIMMs are getting close to their limit as well Essentially, the closer the RAM sits to the CPU, the faster/lower latency/lower power it gets. This is especially important in graphics heavy chips, where the wide, high frequency busses can eat loads of power. In fact, I wouldn't be surprised if Apple transitions to something "new" like a newer generation of Samsung's Wide I/O if they can get memory vendors to produce it in bulk.
DannyBee 3y ago
It's remarkably difficult to maintain signal integrity/etc if it's not very close to the CPU at the signal rates they are getting to. This is already a serious problem. For places where you are willing to give up power efficiency, or latency, or both, this is not as big a deal - this is why you see CXL/etc. But for a laptop, where people want power efficiency, latency, etc, this is a serious issue.
- deleted 3y ago
  [deleted]
sliken 3y ago
Lets compare. The m3 max has up to 128GB of ram, 400GB/sec bandwidth, and uses that memory for both CPU and GPU. It fits in thin (under an inch thick) laptops and is efficient enough to have great battery life on the order of 10-20 hours depending on use cases. So efficient that performance is identical when running on battery or wall power, unlike many of it's competitors. The power savings is even more impressive once you consider you don't need a discrete GPU for many use cases. The AMD Epyc has 12 channels of DDR5-4800 and manages 460GB/sec. It's a physically large chip, can handle a ton of ram, but also uses a ton of power. You can fit a crippled version (without all the memory channels) on a micro atx motherboard that's several inches thick. Normally you'd need an ATX, EATX, or rack mount motherboard. It needs several 100 watts for just the socket, not to mention the numerous dimms (12 minimum for 12 memory channels). It's the kind of thing you see in a data center, after you put ear protection on. If you compare the M3 max, which fits in a laptop or small desktop, to Intel and AMD competition that fits in a thin laptop you end up with 1/4th the memory bandwidth. The competition either matches the power use and is much slower, or is as fast or faster at a huge power penalty. AMD and Intel laptops need discrete GPUs to compete and often require wall power to run at full speed. All in all the M3 max is an impressive chip, fast, and very efficient. But part of that is the soldered in dimms that allow for great bandwidth and low power use.
- brucethemoose2 3y ago
  This is Apples and oranges. A better AMD comparison is Van Gogh (the Steam Deck chip) to the base M1, and on paper they are very similar. The M1 is just a more "premium" version of the budget AMD chip. AMD could make a good M3 competitor (and maybe even improve on it by tiling the memory controller/cache like the 7900), but... There is just no market for it. Windows OEMs dont want a big expensive SoC, they want a cheap CPU to hang a power hungry graphics card off of. Hence the successor to Van Gogh was canceled because no one wanted it.
  - sliken 3y ago
    AMD/Intel iGPUs stink, mostly because the standard 128 bit wide memory system from the lowest end Intel/AMD laptop/desktop chips to the high end like the Ryzen 9 7950x have the same memory bandwidth. AMD even makes better memory systems for the XboxX and PS5, which run games great. For years gamers drifted from gaming desktops to consoles because GPUs were insanely priced or unavailable. However, finally, many years after the GPU shortage there's GASP going to be a laptop/desktop chip with iGPU with more than a 128 bit wide memory interface. The AMD Strix Halo. I hope it does well, it's roughly similar to the M2 Pro. Still half a M2 Max or 1/4th a M2 ultra.

bryanlarsen 3y ago

M2 pro/max/ultra bandwidth is 200/400/800 GBps. Ryzen dual channel bandwidth is 100GBps. EPYC 8 channel bandwidth is well under 400GBps.

brucethemoose2 3y ago
And more importantly, M2's energy/bit is pretty low.
matja 3y ago
* Shared bandwidth between CPU and GPU - as soon as you use up one, you're bottlenecking the other. * No M2 memory benchmark I've seen gets above ~250GB/s once cache effects are exceeded. * Random access latency is >~110ns which is no improvement over even mid-range DDR4 designs from 5 years ago. TL;DR: It's about cost

brucethemoose2 3y ago

Because the signal integrity across the pins and traces becomes a serious bottleneck. SODIMMs are already "past" the limit, requiring 1.35V for modest DDR5 speeds at terrible timings, hence vendors are trying to migrate to this: https://www.anandtech.com/show/21069/modular-lpddr-becomes-a-reality-samsung-introduces-lpcamm-memory-modules https://www.anandtech.com/show/21069/modular-lpddr-becomes-a... I suspect regular desktop/server DIMMs are getting close to their limit as well Essentially, the closer the RAM sits to the CPU, the faster/lower latency/lower power it gets. This is especially important in graphics heavy chips, where the wide, high frequency busses can eat loads of power. In fact, I wouldn't be surprised if Apple transitions to something "new" like a newer generation of Samsung's Wide I/O if they can get memory vendors to produce it in bulk.

DannyBee 3y ago

It's remarkably difficult to maintain signal integrity/etc if it's not very close to the CPU at the signal rates they are getting to. This is already a serious problem. For places where you are willing to give up power efficiency, or latency, or both, this is not as big a deal - this is why you see CXL/etc. But for a laptop, where people want power efficiency, latency, etc, this is a serious issue.

deleted 3y ago
[deleted]

sliken 3y ago

Lets compare. The m3 max has up to 128GB of ram, 400GB/sec bandwidth, and uses that memory for both CPU and GPU. It fits in thin (under an inch thick) laptops and is efficient enough to have great battery life on the order of 10-20 hours depending on use cases. So efficient that performance is identical when running on battery or wall power, unlike many of it's competitors. The power savings is even more impressive once you consider you don't need a discrete GPU for many use cases. The AMD Epyc has 12 channels of DDR5-4800 and manages 460GB/sec. It's a physically large chip, can handle a ton of ram, but also uses a ton of power. You can fit a crippled version (without all the memory channels) on a micro atx motherboard that's several inches thick. Normally you'd need an ATX, EATX, or rack mount motherboard. It needs several 100 watts for just the socket, not to mention the numerous dimms (12 minimum for 12 memory channels). It's the kind of thing you see in a data center, after you put ear protection on. If you compare the M3 max, which fits in a laptop or small desktop, to Intel and AMD competition that fits in a thin laptop you end up with 1/4th the memory bandwidth. The competition either matches the power use and is much slower, or is as fast or faster at a huge power penalty. AMD and Intel laptops need discrete GPUs to compete and often require wall power to run at full speed. All in all the M3 max is an impressive chip, fast, and very efficient. But part of that is the soldered in dimms that allow for great bandwidth and low power use.

brucethemoose2 3y ago
This is Apples and oranges. A better AMD comparison is Van Gogh (the Steam Deck chip) to the base M1, and on paper they are very similar. The M1 is just a more "premium" version of the budget AMD chip. AMD could make a good M3 competitor (and maybe even improve on it by tiling the memory controller/cache like the 7900), but... There is just no market for it. Windows OEMs dont want a big expensive SoC, they want a cheap CPU to hang a power hungry graphics card off of. Hence the successor to Van Gogh was canceled because no one wanted it.
- sliken 3y ago
  AMD/Intel iGPUs stink, mostly because the standard 128 bit wide memory system from the lowest end Intel/AMD laptop/desktop chips to the high end like the Ryzen 9 7950x have the same memory bandwidth. AMD even makes better memory systems for the XboxX and PS5, which run games great. For years gamers drifted from gaming desktops to consoles because GPUs were insanely priced or unavailable. However, finally, many years after the GPU shortage there's GASP going to be a laptop/desktop chip with iGPU with more than a 128 bit wide memory interface. The AMD Strix Halo. I hope it does well, it's roughly similar to the M2 Pro. Still half a M2 Max or 1/4th a M2 ultra.