Very close to the point where SMT/HyperThreading might be enough, where we can just soak the latency & treat it basically like main memory. I would not be shock

Very close to the point where SMT/HyperThreading might be enough, where we can just soak the latency & treat it basically like main memory. I would not be shocked to see SMT3 or SMT4 show up, once we see massively many core scale out cpus with gobs of memory. Load/stores take longer, so pipelines stall, so you want to be able to keep the core busy by switching to external work. SMT is there to make sure your execution units stay busy. Or maybe the SMT2 that we have now is enough for many workloads even with this. I did some scrounging to see if the pyramid's latency numbers felt right & it's pretty ok on 1p. On an 1p AMD Milan core to core latency across an 8 core CCX is low 20ns latency. Thays tiny! Accessing memory on any other CCX has to go off the CCX to the IOD and back, which is high 80s to 110ns latency. This example is from an aws c6a.metal. https://github.com/nviennot/core-to-core-latency#amd-epyc-7r13-48-cores-milan-3rd-gen-2021-q1 https://github.com/nviennot/core-to-core-latency#amd-epyc-7r... Intel Ice Lake (c6i.metal), being monolithic, starts way worse. Any communication has to traverse a shared ring bus and thus takes 40 minimum. But only peaks at 65ns, I much better than Milan. I https://github.com/nviennot/core-to-core-latency#intel-xeon-platinum-8375c-290ghz-32-cores-ice-lake-3rd-gen-2021-q2 https://github.com/nviennot/core-to-core-latency#intel-xeon-... M1 Pro is neat. An 8c chip has three "CCX" alike complexes, 2 perf of 3c each and 1 efficiency of 2c. Smart. Latency is 40ns across cluster, 150ns outside cluster. https://github.com/nviennot/core-to-core-latency#apple-m1-pro-6p2e-cores-2021-q4 https://github.com/nviennot/core-to-core-latency#apple-m1-pr... Doing anything off the first socket on AMD is terrible. 90-110ns across CCX on the same CCD, but any communication involving the 2nd CCD at all (even across the CCD itself) is a staggering 190ns to 210ns. https://github.com/nviennot/core-to-core-latency#dual-amd-epyc-7r13-48-cores-milan-3rd-gen-2021-q1 https://github.com/nviennot/core-to-core-latency#dual-amd-ep... That's around what the pyramid shows as the upper end for CXL memory (170-250ns)! Please also kindly note these figures probably don't scale linearly with core clockspeed but they probably do scale somewhat; direct comparison is inadvised. But it's good interesting directional data showing some very contemporary latency situations deep in the heart of computing that CXL is unlikely to be better than. Using core to core is a weird proxy but illuminative of how complex & odd it is providing system connectivity is to the smaller CCX core clusters. More on point is talking about main memory latency. Anandtech has great coverage of core to core, and also crucially main memory latency too. There's a lot of nuance & config variance here (NPS0-4) but there's generally a regime where a cluster can be getting around 12ns access (oh that's L3 cache), but it will quickly ramp up to 110-140ns if trying to access wide ranges of memory. It starts to look like a core to core grade speed hit. (Anandtech doesn't cover cross socket memory but it won't be good!) https://www.anandtech.com/show/16529/amd-epyc-milan-review/4 https://www.anandtech.com/show/16529/amd-epyc-milan-review/4 Notably the IOD is basically a Northbridge controller connecting all the individual CCX clusters: key for the talking to other clusters, key to the to talking to memory, key to the exposing PCIe/CXL. If core to core is 150ns say, it's well possible CXL's additional overhead could actually be quite marginal! That's what the article seems to be arguing. Maybe, or not, maybe much penalty will be on top of this hit; too early to tell probably. Damn computing architectures are neat. CXL will add so much new ability to try new things. My main sadness though is just that Nvidia now has 900GBps switched fabric that vastly out stages CXL, which still hasn't shipped 3.0, which is the first switched CXL (aka multiple hosts can share the CXL network); a hypothetical PCIe 6.0 x16 based CXL implementation is comparatively only 121GBps, doesn't exist yet, and isn't switched yet. Then again, AMD EPYC sockets are 128 lane (in 1p mode), so that actually does aggregate to almost 2TBps connectivity... I ran numbers recently & it was not this sunny: I think I'd miscalculated somehow! Very excited for new possibilities. An ecosystem that people can build parts for & try things on, at these great capacities, being near to cores, will be compelling. I hope RISC-V can come play too, ideally with some open source designs (albeit right now everyone is mostly vendored up for their PCIe but CXL hypothetically is simpler).