To expand a bit, I did R&D for a supercomputing companies on software latency-hiding many years ago. The idea was that we could run many of these codes on commo

To expand a bit, I did R&D for a supercomputing companies on software latency-hiding many years ago. The idea was that we could run many of these codes on commodity hardware by taking ideas from exotic latency-hiding silicon, which had no direct analogues that could be implemented in software, and inventing something similar for software. This was surprisingly successful! Unfortunately, this had two practical problems. First, it turns out that developers are quite poor at reasoning about control flow in latency-hiding architectures generally. It is analogous to reasoning about very complex lock graphs but worse. Second, you still need prodigious quantities of high-quality network bandwidth and topology, even if latency matters much less, for typical HPC problems. At which point you are half the way to a traditional HPC network anyway. There is still a space for this research in non-HPC applications (like join parallelization in databases), but for HPC the cost-benefit ratio pushes everyone to purpose-built networks. Less learning curve for the devs and you get the exceptional bandwidth and topology the codes need anyway.