3 ms·
HBM has such high access latency that the Xeon Max can't actually use all of the available bandwidth except in a caching mode (where a lot of that bandwidth is
by pclmulqdq 3y ago
HBM has such high access latency that the Xeon Max can't actually use all of the available bandwidth except in a caching mode (where a lot of that bandwidth is used with cache line spills and fills): the cores' line fill buffers will get full and cause backpressure and stalls in the CPU. And this assumes that every core is trying to load all the time.
You can see line fill buffer issues with 8-socket machines today, for example, where they are the cause of stalls due to the high latency of cache coherency checks. However, since they are part of the core and fine for 99% of users, the size of the line fill buffer is kept relatively small.
That limitation puts the HBM Xeon Max sort of on par with Grace for actual usable memory bandwidth, not so far above it.
- anonymousDan 3y agoThat's super interesting, can you by any chance point to a paper or article with benchmarking results?
- moggi 3y agoA really good presentation showing why the Xeon Max can't reach full HBM bandwidth: https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limits_2023-05-24_final.pdf https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limi...
- sliken 3y agoInteresting paper. I focus more on the latency than bandwidth. The paper gets a few things wrong, DDR5 is not a single 64bit channel, but 2 x 32 bit channels. So the normal Xeon is 16 x 32 bit channels, not 8x64 bit. He talks about cache misses as 60ns, which glosses over that approximately half of that is missing through L1/L2/L3, then you enter the queue for the memory controller, for the memory channel you need. As a result you only get half the bandwidth if you only have a single request pending per channel.
- Remnant44 3y agoThat's a great paper. I've tangled with the line fill buffer issue before, along with other hidden limits such as infinity fabric limits on AMD systems. It's one of the first big disppointments when you start doing high performance, bandwidth-limited compute work. It's also one that's almost entirely undocumented which is frustrating.