4 ms·

TOP500 hasn't been a particularly useful measure of practical computing power in modern systems for many years because what it measures isn't a significant bott

TOP500 hasn't been a particularly useful measure of practical computing power in modern systems for many years because what it measures isn't a significant bottleneck in most real systems. It has become a measure of how much money someone is willing to spend for bragging rights. (HPCG is better in that it is a bit more bandwidth focused but still pretty narrow.) Most companies with huge systems don't participate.

bee_rider 4mo ago
I wonder if there would have been an opportunity to generate some finer-grained benchmarks with something like BiCGStab+ILU (or maybe CG+incomplete cholesky). Instead of CG+Gauss Seidel. The pitch being, you might have made different memory vs compute trade-offs with designing your cluster, but you should be able to select a fill-in factor for the preconditioner to suit it.
- jandrewrogers 4mo ago
  I think you could build more representative benchmarks that capture capability better. The tension is that HPC companies are pure hardware companies and you need a lot of help from software to make your hardware look good. They don’t like that. Most of our software sucks at the scales they try to benchmark. Ironically, the related Graph500 benchmarks reflect this better. Performance is dependent more on using the hardware better than better hardware per se.
adrian_b 4mo ago
This seems like a "sour grapes" comment. The new Chinese supercomputer beats all US supercomputers also in HPCG, not only in Linpack. What is remarkable is that this was done despite the US attempts of sabotaging HPC in China by "sanctions". This uses custom CPUs designed in China, which implement an Armv9-A ISA with SME (scalable matrix extension) and which use fast HBM memory. These CPUs are fast enough that they do not need any GPUs for exceeding the throughput of the American supercomputers, which use GPUs. This is like in the Japanese Fugaku, which was the first to implement the Armv8-A ISA with SVE, but which now is rather old. Like in all CPU-based supercomputers, for this new Chinese supercomputer it is much easier to reach a higher percentage of the theoretical maximum throughput, when solving any problem. So for most practical problems it will be faster than a GPU-based supercomputer that would have the same theoretical maximum throughput. So this is a much more interesting supercomputer than those built by just buying some HPC racks from HPE (Cray). Because China was forbidden to buy the American equipment, they had to innovate and design their own. Eventually they made something better than what they could not buy.
- jandrewrogers 4mo ago
  It isn’t “sour grapes”, I remember when the HPC community largely abandoned these benchmarks two decades ago because they weren’t representative of anything real for most of them. The benchmark is a poor reflection of real workloads. There was a long period when the STREAM benchmark was the primary correlate with real-world performance for most HPC workloads but you can’t build a press release from that. I don’t have a dog in this fight and I no longer work in HPC. Most modern workloads are severely bandwidth bound. The only aspect of the hardware that matters is bandwidth and that is not materially differentiated. The frontier is scheduler design, which is pure software and difficult computer science. HPC competitions avoid problems with a software solution because it isn’t in their interest as hardware manufacturers. This result is impressive, sort of, but not in the way people are imagining. I was equally dismissive of the previous leader for the same reasons. For most applications, these benchmarks are legacy pagentry.
  - adrian_b 4mo ago
    As I have mentioned, and as described in TFA, this supercomputer is also leading in memory bandwidth (4 TB/s per socket => correction, it is 8 Tb/s per socket). I agree with what you say about benchmarks, but that is precisely why the advantage of this supercomputer over the following American supercomputers will be even greater in more demanding workloads than Linpack and HPCG. It has already shown this by having an advantage in HPCG of greater than 26% over the fastest US system, while in Linpack its advantage is of only 22%. Thus its position in the top cannot be dismissed as insignificant, because it more likely underestimates than overestimates this system. Also the programming effort for writing an efficient program will be lower than for the GPU-based US supercomputers.
    - jandrewrogers 4mo ago
      I definitely appreciate the CPU-centric approach. It appeals to my biases and aligns with my technical perspective. The memory bandwidth is something you can buy. Exotics were >1 TB/s over a decade ago, so 4 TB/s in 2026 is not that impressive. For all practical purposes, these CPUs are also still exotics, you can’t just buy them. I would be very surprised if the memory bandwidth of US exotics haven’t improved over the last 10-15 years. In any case, for real workloads scalability is mostly a software theory problem at this point and that is still a dark art without much literature.

bee_rider 4mo ago

I wonder if there would have been an opportunity to generate some finer-grained benchmarks with something like BiCGStab+ILU (or maybe CG+incomplete cholesky). Instead of CG+Gauss Seidel. The pitch being, you might have made different memory vs compute trade-offs with designing your cluster, but you should be able to select a fill-in factor for the preconditioner to suit it.

jandrewrogers 4mo ago
I think you could build more representative benchmarks that capture capability better. The tension is that HPC companies are pure hardware companies and you need a lot of help from software to make your hardware look good. They don’t like that. Most of our software sucks at the scales they try to benchmark. Ironically, the related Graph500 benchmarks reflect this better. Performance is dependent more on using the hardware better than better hardware per se.

adrian_b 4mo ago

This seems like a "sour grapes" comment. The new Chinese supercomputer beats all US supercomputers also in HPCG, not only in Linpack. What is remarkable is that this was done despite the US attempts of sabotaging HPC in China by "sanctions". This uses custom CPUs designed in China, which implement an Armv9-A ISA with SME (scalable matrix extension) and which use fast HBM memory. These CPUs are fast enough that they do not need any GPUs for exceeding the throughput of the American supercomputers, which use GPUs. This is like in the Japanese Fugaku, which was the first to implement the Armv8-A ISA with SVE, but which now is rather old. Like in all CPU-based supercomputers, for this new Chinese supercomputer it is much easier to reach a higher percentage of the theoretical maximum throughput, when solving any problem. So for most practical problems it will be faster than a GPU-based supercomputer that would have the same theoretical maximum throughput. So this is a much more interesting supercomputer than those built by just buying some HPC racks from HPE (Cray). Because China was forbidden to buy the American equipment, they had to innovate and design their own. Eventually they made something better than what they could not buy.

jandrewrogers 4mo ago
It isn’t “sour grapes”, I remember when the HPC community largely abandoned these benchmarks two decades ago because they weren’t representative of anything real for most of them. The benchmark is a poor reflection of real workloads. There was a long period when the STREAM benchmark was the primary correlate with real-world performance for most HPC workloads but you can’t build a press release from that. I don’t have a dog in this fight and I no longer work in HPC. Most modern workloads are severely bandwidth bound. The only aspect of the hardware that matters is bandwidth and that is not materially differentiated. The frontier is scheduler design, which is pure software and difficult computer science. HPC competitions avoid problems with a software solution because it isn’t in their interest as hardware manufacturers. This result is impressive, sort of, but not in the way people are imagining. I was equally dismissive of the previous leader for the same reasons. For most applications, these benchmarks are legacy pagentry.
- adrian_b 4mo ago
  As I have mentioned, and as described in TFA, this supercomputer is also leading in memory bandwidth (4 TB/s per socket => correction, it is 8 Tb/s per socket). I agree with what you say about benchmarks, but that is precisely why the advantage of this supercomputer over the following American supercomputers will be even greater in more demanding workloads than Linpack and HPCG. It has already shown this by having an advantage in HPCG of greater than 26% over the fastest US system, while in Linpack its advantage is of only 22%. Thus its position in the top cannot be dismissed as insignificant, because it more likely underestimates than overestimates this system. Also the programming effort for writing an efficient program will be lower than for the GPU-based US supercomputers.
  - jandrewrogers 4mo ago
    I definitely appreciate the CPU-centric approach. It appeals to my biases and aligns with my technical perspective. The memory bandwidth is something you can buy. Exotics were >1 TB/s over a decade ago, so 4 TB/s in 2026 is not that impressive. For all practical purposes, these CPUs are also still exotics, you can’t just buy them. I would be very surprised if the memory bandwidth of US exotics haven’t improved over the last 10-15 years. In any case, for real workloads scalability is mostly a software theory problem at this point and that is still a dark art without much literature.