Ah, thanks, i completely overlooked that mention of HBM2E. So one stack of HBM2E can hold 24 GB, accessed at 410 GB/s. And we can have multiple stacks per pack

Ah, thanks, i completely overlooked that mention of HBM2E. So one stack of HBM2E can hold 24 GB, accessed at 410 GB/s. And we can have multiple stacks per package - i see lots of diagrams with four stacks. So a plausible scenario is 80 cores with 96 GB and 1640 GB/s of HBM2E. Per core, that's 1.2 GB and 20.5 GB/s. That's ~320 million cache line fills per second. If the clock speed is 4.6 GHz, that's a cache line fill every 15 cycles, or a 64-bit load every 1.8 cycles. That seems pretty good!