6 ms·

I couldn't figure out what the actual measured memory bandwidth to the onboard HBM2e memory chiplets are- how does it compare to, say, Apple's M2 Ultra?

JacobiX 3y ago
The M1 Ultra has 800GB/s of memory bandwidth, on contrast HBM2E has 204.8 Gbps × 2 = 409.6 Gb/s
- consp 3y ago
  Is that available to any core or just a sum-all-up and has latency penalties when going around?
  - hypercube33 3y ago
    depends on NUMA node config so I believe this is combined on the whole chip if all cores are working on threads with their local 16GB HBM, theoretically.
- Brian-Puccio 3y ago
  So 6,400 Gb/s compared to 409.6 Gb/s once you convert units?
- wmf 3y ago
  GPUs use 4-6 stacks of HBM2 which is 1,840-2,760 GB/s. It's 2x-3x the bandwidth of M1/M2 Ultra.
opencl 3y ago
According to this paper[0] ~1600GB/s raw memory bandwidth, ~590GB/s in practice. I haven't seen any actual benchmarks of M2 Ultra CPU bandwidth, but the M1 Ultra has been benchmarked at ~240GB/s[1]. [0] https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limits_2023-05-24_final.pdf https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limi... [1] https://macperformanceguide.com/blog/2022/20220618_1934-AppleMacStudioM1Ultra-tested-MemoryBandwidth.html https://macperformanceguide.com/blog/2022/20220618_1934-Appl...
- doctorpangloss 3y ago
  > The cores used in the Xeon Max processors do not support enough concurrency to reach full HBM bandwidth with the current core counts. 2x more concurrency or 2x more cores. is a good punchline from the report. The Apple cores have that problem but a lot worse. They are slow. It goes back to this flawed idea in the community that you can "just" "add" "more memory," when parts like the H100 have their memory size matched to the architecture (physical and software) of the dozens of CPUs on them. I'm not sure why the conception persists that a 60W laptop part would be comparable to a 300W server part of the same process generations, let alone this particular part.
- foota 3y ago
  That's insane.
- deleted 3y ago
  [deleted]

JacobiX 3y ago

The M1 Ultra has 800GB/s of memory bandwidth, on contrast HBM2E has 204.8 Gbps × 2 = 409.6 Gb/s

consp 3y ago
Is that available to any core or just a sum-all-up and has latency penalties when going around?
- hypercube33 3y ago
  depends on NUMA node config so I believe this is combined on the whole chip if all cores are working on threads with their local 16GB HBM, theoretically.
Brian-Puccio 3y ago
So 6,400 Gb/s compared to 409.6 Gb/s once you convert units?
wmf 3y ago
GPUs use 4-6 stacks of HBM2 which is 1,840-2,760 GB/s. It's 2x-3x the bandwidth of M1/M2 Ultra.

opencl 3y ago

According to this paper[0] ~1600GB/s raw memory bandwidth, ~590GB/s in practice. I haven't seen any actual benchmarks of M2 Ultra CPU bandwidth, but the M1 Ultra has been benchmarked at ~240GB/s[1]. [0] https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limits_2023-05-24_final.pdf https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limi... [1] https://macperformanceguide.com/blog/2022/20220618_1934-AppleMacStudioM1Ultra-tested-MemoryBandwidth.html https://macperformanceguide.com/blog/2022/20220618_1934-Appl...

doctorpangloss 3y ago
> The cores used in the Xeon Max processors do not support enough concurrency to reach full HBM bandwidth with the current core counts. 2x more concurrency or 2x more cores. is a good punchline from the report. The Apple cores have that problem but a lot worse. They are slow. It goes back to this flawed idea in the community that you can "just" "add" "more memory," when parts like the H100 have their memory size matched to the architecture (physical and software) of the dozens of CPUs on them. I'm not sure why the conception persists that a 60W laptop part would be comparable to a 300W server part of the same process generations, let alone this particular part.
foota 3y ago
That's insane.
deleted 3y ago
[deleted]