3 ms·
Did you have an embarassingly parallel code (like, "map"), or a code that was ported to supercomputers? I've worked on supercomputers for years and generally,
by dekhn 7y ago
Did you have an embarassingly parallel code (like, "map"), or a code that was ported to supercomputers?
I've worked on supercomputers for years and generally, one could not run code at scale unless it scaled- in terms of parallel performance- and also used the network in a non-trivial way. Supercomputers aren't just speedups for normal codes (I'm pretty sure you know this).
The real issue with modern supercomputers is that basically none of them have decent disk IO. That's what differs between modern Internet/cloud clusters and supercomputers. Cloud clusters emphasize very high connectivity between durable storage and worker nodes. None of the supercomputers have a decent disk IO stack (mostly GPFS and Lustre) and this ends up being no end of pain for applicaiton developers. The only recent improvement in this area was "burst buffers", but that's really just accelerated data staging.