4 ms·

I generally expect Grace to show up as underpowered compared to modern AMD and Intel offerings (Skylake is old), but if they do end up with any wins when they r

I generally expect Grace to show up as underpowered compared to modern AMD and Intel offerings (Skylake is old), but if they do end up with any wins when they run that benchmark, I would assume it would be due to the memory bandwidth. LPDDR5 on a server chip with 2x the bandwidth of the DDR5-based competitors is really good to see.

jeffbee 3y ago
Wouldn't the HBM-equipped Xeon Max be the bandwidth king of these tables?
- pclmulqdq 3y ago
  HBM has such high access latency that the Xeon Max can't actually use all of the available bandwidth except in a caching mode (where a lot of that bandwidth is used with cache line spills and fills): the cores' line fill buffers will get full and cause backpressure and stalls in the CPU. And this assumes that every core is trying to load all the time. You can see line fill buffer issues with 8-socket machines today, for example, where they are the cause of stalls due to the high latency of cache coherency checks. However, since they are part of the core and fine for 99% of users, the size of the line fill buffer is kept relatively small. That limitation puts the HBM Xeon Max sort of on par with Grace for actual usable memory bandwidth, not so far above it.
  - anonymousDan 3y ago
    That's super interesting, can you by any chance point to a paper or article with benchmarking results?
    - moggi 3y ago
      A really good presentation showing why the Xeon Max can't reach full HBM bandwidth: https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limits_2023-05-24_final.pdf https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limi...
      - sliken 3y ago
        Interesting paper. I focus more on the latency than bandwidth. The paper gets a few things wrong, DDR5 is not a single 64bit channel, but 2 x 32 bit channels. So the normal Xeon is 16 x 32 bit channels, not 8x64 bit. He talks about cache misses as 60ns, which glosses over that approximately half of that is missing through L1/L2/L3, then you enter the queue for the memory controller, for the memory channel you need. As a result you only get half the bandwidth if you only have a single request pending per channel.
      - Remnant44 3y ago
        That's a great paper. I've tangled with the line fill buffer issue before, along with other hidden limits such as infinity fabric limits on AMD systems. It's one of the first big disppointments when you start doing high performance, bandwidth-limited compute work. It's also one that's almost entirely undocumented which is frustrating.
Remnant44 3y ago
Impressive memory bandwidth! I feel like x86 offerings often fall down so flat in this area. Even when theoretical memory bandwidth is provided, there are often other significant limitations that prevent its use. I've run into this personally with AMD Zen cores, where it is pretty easy to saturate the infinite fabric bandwidth. Do you know why they would outfit it with LPDDR5 instead of regular DDR5? I don't have a good feel for the difference here. At first blush I would assume low power DDR would run slower not faster, but that doesn't seem to be the case anywhere I see it deployed.
- Vecr 3y ago
  LPDDR5 has lower total latency (or can, and I assume it very much does in this case). Essentially the "standard motherboard layout" normal DDR5 is designed for is too big to target a lower latency.

jeffbee 3y ago

Wouldn't the HBM-equipped Xeon Max be the bandwidth king of these tables?

pclmulqdq 3y ago
HBM has such high access latency that the Xeon Max can't actually use all of the available bandwidth except in a caching mode (where a lot of that bandwidth is used with cache line spills and fills): the cores' line fill buffers will get full and cause backpressure and stalls in the CPU. And this assumes that every core is trying to load all the time. You can see line fill buffer issues with 8-socket machines today, for example, where they are the cause of stalls due to the high latency of cache coherency checks. However, since they are part of the core and fine for 99% of users, the size of the line fill buffer is kept relatively small. That limitation puts the HBM Xeon Max sort of on par with Grace for actual usable memory bandwidth, not so far above it.
- anonymousDan 3y ago
  That's super interesting, can you by any chance point to a paper or article with benchmarking results?
  - moggi 3y ago
    A really good presentation showing why the Xeon Max can't reach full HBM bandwidth: https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limits_2023-05-24_final.pdf https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limi...
    - sliken 3y ago
      Interesting paper. I focus more on the latency than bandwidth. The paper gets a few things wrong, DDR5 is not a single 64bit channel, but 2 x 32 bit channels. So the normal Xeon is 16 x 32 bit channels, not 8x64 bit. He talks about cache misses as 60ns, which glosses over that approximately half of that is missing through L1/L2/L3, then you enter the queue for the memory controller, for the memory channel you need. As a result you only get half the bandwidth if you only have a single request pending per channel.
    - Remnant44 3y ago
      That's a great paper. I've tangled with the line fill buffer issue before, along with other hidden limits such as infinity fabric limits on AMD systems. It's one of the first big disppointments when you start doing high performance, bandwidth-limited compute work. It's also one that's almost entirely undocumented which is frustrating.

Remnant44 3y ago

Impressive memory bandwidth! I feel like x86 offerings often fall down so flat in this area. Even when theoretical memory bandwidth is provided, there are often other significant limitations that prevent its use. I've run into this personally with AMD Zen cores, where it is pretty easy to saturate the infinite fabric bandwidth. Do you know why they would outfit it with LPDDR5 instead of regular DDR5? I don't have a good feel for the difference here. At first blush I would assume low power DDR would run slower not faster, but that doesn't seem to be the case anywhere I see it deployed.

Vecr 3y ago
LPDDR5 has lower total latency (or can, and I assume it very much does in this case). Essentially the "standard motherboard layout" normal DDR5 is designed for is too big to target a lower latency.