5 ms·

> much more usable than similar amounts of ram People keep repeating this but how does higher bandwidth (probably not 8x higher though) compensate for a lower

> much more usable than similar amounts of ram People keep repeating this but how does higher bandwidth (probably not 8x higher though) compensate for a lower amount of RAM? It's not quite as silly as the people saying that 8GB in the base config 'feels' much faster than 8GB on a PC cause the drive/swap are "so fast" but still..

lambdasquirrel 3y ago
Maybe it adds another step to the memory mountain? Folks who use these kinds of workstations might think of in-package RAM as just the next level of cache, if someone goes ahead and makes a card with comparatively slower memory card slots.
dathinab 3y ago
The DDR5 bandwidth is ~51.2 GiB/s per "channel". With consumer desktop CPUs having 2 "channels" (~102.4GiB/s). And prosumer desktop/workstation CPUs (e.g. Threadripper) having 4 "channels" (~204.8GiB/s). While apple is claiming 819.2GiB/s. That is _max throughput is 4x/8x more_ (depending if you compare it to prosumer (fair comparison) or consumer (unfair comparison for a $7k system) hardware. Now the _max_ part is important, Apple mainly reaches this by having more bandwidth, i.e. parallelism. Mainly (oversimplified) the per "CPU memory channel" bandwidth for x86 is 64bit (for 51.2GiB/s) so the M1 Ultra is roughly comparable to having 16 "CPU memory channels" instead of 8 (prosumer) or 4 (consumer). Some applications can take advantage of this nicely and will scale potentially even to 4x/8x speed, most probably will not some might even have neglible improvements. But applications which use a lot of RAM (as much as they can get) and most of the RAM they use is "warm" (e.g. doesn't just lie around with little access but isn't super "hot", i.e. highly contented either) will profit quite a lot. On the other hand applications which use little RAM but the same small RAM region very heavily contented likely will hardly at all profit and will likely run faster on overclocked RAM on consumer systems. Luckily for apple most the typical use case they sell their pro desktop model for belong more/mostly in the first category. Additionally if you can run Linux on this system they might become _very_ interesting for some scientific applications for some users. I mean even e.g. Zen 4 EPYC CPU only have 12 "CPU memory channels" not 16 and it's much easier to put a desktop box "somewhere" then a server unit. Side note: I say "CPU channel" in quotes because while it tends to be the marketing term things are more complicated in practice, e.g. in general DDR5 is splitting the 64bit channels into 32bit sub-channels, and just listing the channel width and throughput is still not painting the whole picture at all, e.g. the latency also matters for some applications (hence why OC can make sense) etc. EDIT: Correction: The Threadripper PRO models have 8 "CPU memory channels" so it's just 2x on a fair comparison.
- rektide 3y ago
  AMD's Strix Halo is an APU targeted next year that supposedly has 256-bit wide LPDDR5X memory, on laptops I think, which should be nice. I do hope folks other than Apple can get fast! The new CAMM dual channel module will hopefully help reduce footprint but there's plenty of soldered down systems so it's not really required. It also surprises me there's not a GDDR based APU, except I guess too many games need both a bunch of system ram and video ram and 32GB vram is expensive and has significant lower draw. Apple going wide is really the obvious move, & doing it on package was the best way to do it, it seems.
  - dathinab 3y ago
    > not a GDDR based APU AFIK Apple currently doesn't have a foodhold on gaming outside of phone/tablet games (where they are strong, but the games are used to LPDDR perf.). And while many of the Graphic applications people do use would profit from it I'm not sure it's that big of benefit. In the end a lot of users just do daily tasks on Apple laptops and for that battery life absolutely trumps any speed benefits GDDR gives. I guess the desktop versions could/should use GDDR, but there are two issue: 1. Heat, 2. more design/variance in the CPU production supply chain. The 2nd can drive up cost by quite a surprising high amount. The 1st point I'm not supper sure about. But AFIK one big problem with things which are stacked on-die is heat as all the heat of the CPU needs to go through whatever is stacked onto it to reach the cooler. And while there are probably all kinds of tricks to improve this I could imagine that using a GDDR which has a higher power draw and produces more heat itself could make this more of an issue. But that is purely speculative. I guess we have to see until around 2025+ to know if such stacked chips are more prone to die a early (i.e. <5 years) heat death. Especially some of the Air models without heat pipes could be at risk if used in a less climate controlled environment. Or it could be all perfectly fine. I'm looking forward to finding out.
    - rektide 3y ago
      Apple already outclasses everyone in throughput & throughput/watt especially. In case it wasn't already crystal clear, I was talking about everyone else finding ways to get their throughput up to more competitive levels, such as by supporting GDDR.
rodgerd 3y ago
> People keep repeating this but how does higher bandwidth (probably not 8x higher though) compensate for a lower amount of RAM? I guess it depends on your use case, but back when part of my day job was debugging performance problems with JVM-hosted applications, one of the things that was most noticeable was that the degree to which latency blows up memory use - whether the latency was GC, disk, network, DB queries, whatever. It all turns into holding items in memory longer before they get processed (which in the case of older JVMs, turns into a death spiral of GC thrashing, which blows up processing times further, until your application is staggering along). Increasing memory can alleviate this, but only up to a point - and it can make things worse, because you've now got significant overhead managing the in-flight workloads. It's also possibly that this is simply a decision driven by what Apple can produce with their M2 chips at this point, and that they wanted to offer 384 or 640 GB as the maximum, and everyone is making excuses for them.

lambdasquirrel 3y ago

Maybe it adds another step to the memory mountain? Folks who use these kinds of workstations might think of in-package RAM as just the next level of cache, if someone goes ahead and makes a card with comparatively slower memory card slots.

dathinab 3y ago

The DDR5 bandwidth is ~51.2 GiB/s per "channel". With consumer desktop CPUs having 2 "channels" (~102.4GiB/s). And prosumer desktop/workstation CPUs (e.g. Threadripper) having 4 "channels" (~204.8GiB/s). While apple is claiming 819.2GiB/s. That is _max throughput is 4x/8x more_ (depending if you compare it to prosumer (fair comparison) or consumer (unfair comparison for a $7k system) hardware. Now the _max_ part is important, Apple mainly reaches this by having more bandwidth, i.e. parallelism. Mainly (oversimplified) the per "CPU memory channel" bandwidth for x86 is 64bit (for 51.2GiB/s) so the M1 Ultra is roughly comparable to having 16 "CPU memory channels" instead of 8 (prosumer) or 4 (consumer). Some applications can take advantage of this nicely and will scale potentially even to 4x/8x speed, most probably will not some might even have neglible improvements. But applications which use a lot of RAM (as much as they can get) and most of the RAM they use is "warm" (e.g. doesn't just lie around with little access but isn't super "hot", i.e. highly contented either) will profit quite a lot. On the other hand applications which use little RAM but the same small RAM region very heavily contented likely will hardly at all profit and will likely run faster on overclocked RAM on consumer systems. Luckily for apple most the typical use case they sell their pro desktop model for belong more/mostly in the first category. Additionally if you can run Linux on this system they might become _very_ interesting for some scientific applications for some users. I mean even e.g. Zen 4 EPYC CPU only have 12 "CPU memory channels" not 16 and it's much easier to put a desktop box "somewhere" then a server unit. Side note: I say "CPU channel" in quotes because while it tends to be the marketing term things are more complicated in practice, e.g. in general DDR5 is splitting the 64bit channels into 32bit sub-channels, and just listing the channel width and throughput is still not painting the whole picture at all, e.g. the latency also matters for some applications (hence why OC can make sense) etc. EDIT: Correction: The Threadripper PRO models have 8 "CPU memory channels" so it's just 2x on a fair comparison.

rektide 3y ago
AMD's Strix Halo is an APU targeted next year that supposedly has 256-bit wide LPDDR5X memory, on laptops I think, which should be nice. I do hope folks other than Apple can get fast! The new CAMM dual channel module will hopefully help reduce footprint but there's plenty of soldered down systems so it's not really required. It also surprises me there's not a GDDR based APU, except I guess too many games need both a bunch of system ram and video ram and 32GB vram is expensive and has significant lower draw. Apple going wide is really the obvious move, & doing it on package was the best way to do it, it seems.
- dathinab 3y ago
  > not a GDDR based APU AFIK Apple currently doesn't have a foodhold on gaming outside of phone/tablet games (where they are strong, but the games are used to LPDDR perf.). And while many of the Graphic applications people do use would profit from it I'm not sure it's that big of benefit. In the end a lot of users just do daily tasks on Apple laptops and for that battery life absolutely trumps any speed benefits GDDR gives. I guess the desktop versions could/should use GDDR, but there are two issue: 1. Heat, 2. more design/variance in the CPU production supply chain. The 2nd can drive up cost by quite a surprising high amount. The 1st point I'm not supper sure about. But AFIK one big problem with things which are stacked on-die is heat as all the heat of the CPU needs to go through whatever is stacked onto it to reach the cooler. And while there are probably all kinds of tricks to improve this I could imagine that using a GDDR which has a higher power draw and produces more heat itself could make this more of an issue. But that is purely speculative. I guess we have to see until around 2025+ to know if such stacked chips are more prone to die a early (i.e. <5 years) heat death. Especially some of the Air models without heat pipes could be at risk if used in a less climate controlled environment. Or it could be all perfectly fine. I'm looking forward to finding out.
  - rektide 3y ago
    Apple already outclasses everyone in throughput & throughput/watt especially. In case it wasn't already crystal clear, I was talking about everyone else finding ways to get their throughput up to more competitive levels, such as by supporting GDDR.

rodgerd 3y ago

> People keep repeating this but how does higher bandwidth (probably not 8x higher though) compensate for a lower amount of RAM? I guess it depends on your use case, but back when part of my day job was debugging performance problems with JVM-hosted applications, one of the things that was most noticeable was that the degree to which latency blows up memory use - whether the latency was GC, disk, network, DB queries, whatever. It all turns into holding items in memory longer before they get processed (which in the case of older JVMs, turns into a death spiral of GC thrashing, which blows up processing times further, until your application is staggering along). Increasing memory can alleviate this, but only up to a point - and it can make things worse, because you've now got significant overhead managing the in-flight workloads. It's also possibly that this is simply a decision driven by what Apple can produce with their M2 chips at this point, and that they wanted to offer 384 or 640 GB as the maximum, and everyone is making excuses for them.