> "the M1 Max isn’t able to fully saturate the SoC bandwidth from just the CPU side Sure, wouldn't want to starve the GPU when the CPU is busy. Not sure if th

> "the M1 Max isn’t able to fully saturate the SoC bandwidth from just the CPU side Sure, wouldn't want to starve the GPU when the CPU is busy. Not sure if the AMX has it's own connection or shares the CPU complex bandwidth. > I guarantee you Xeons and Epycs are not limited to 1 transaction in flight, nor is Graviton2 Right, I said 1 per channel, not 1. I've tested this on Xeons and Epycs, but not genoa yet. Generally maximum random throughput tends to be with 16 or so cache misses per socket with 8 channels. From what I can tell about half the latency is cache misses in L1, L2, and L3. Then you queue in the memory controller and wait for whatever memory channel (or two depending on config). By keeping 16 in the queue as soon as one of the 8 channels free you have another request pending for that channel, at least most of the time. The Genoa looks particularly promising on this front, they upgraded from 8 channels to 12, but the DDR5 dimms actually provide two channels per dimm, so you end up with 24 narrow (32 bit) DDR5 channels. I've already seen results showing that Genoa scales better to high core counts than the zen3 Epyycs, which makes sense with 24 memory requests in flight instead of 8. > M1 ultra is 32 channels. Well, it's really 2x 16 channels since it's a NUMA configuration, like a dual-socket system. Heh, well monolithic chips are on the way out. Even the zen2 epycs used multiple pieces of silicon (chiplets) and the new Intel Xeon/Sapphire Rapids do the same. Don't see a particular difference. Even the desktop Ryzens have multiple pieces of silicon (one IOD and one CPU chiplet at the minimum). In any case the Apple M1 Max memory system matches the Genoa and beats the newest Intel Sapphire Rapids with a single piece of silicon, and wins handily doubles that with two pieces of silicon, unless you wait for the unreleased HBM flavors. Compared to 100% of Intel/AMD laptops and probably 98% of desktops that apple memory system is pretty amazing. Sure there are threadrippers (4 channel), threadripper pro (8 channel), and some similar Intel options, but all are expensive, low volume, and desktop/workstation only.