4 ms·

That's the smallest of 4 experiments. It goes upto 140 million cells, where MI300X retains similar performance advantage of around 10% over Nvidia's H100.

by shihab 2y ago

semi-extrinsic 2y ago
Yeah, but the lack of comparison to a CPU-only system is suspicious. (As is the exaggerated plot without ticks and labels on the y-axis.) I've never seen a non-reactive incompressible flow simulation get substantial speedup on GPUs. There are well understood fundamental reasons why this is the case.
- shihab 2y ago
  I helped develop a hydro solver coupled with radiation at Los Alamos this summer. We observed from 7x upto 15x performance improvement on a single GPU compared to 64-core intel CPU. [1] Now granted, the flops to byte ratio for this program might be better than an avg fluid simulator. Also, our performance tanked when we moved to multi-node system. But I am aware of underlying reasons behind the scalibility issues and they don't feel like problems that can't be overcome. [1] https://github.com/lanl/HARD/tree/main https://github.com/lanl/HARD/tree/main
  - semi-extrinsic 2y ago
    Nice, interesting solver! But yes, compressible/density-based solvers work a lot better on GPUs than incompressible/pressure-based solvers. When 80% of the runtime is spent solving a Poisson equation, GPU is not worth it. Especially if you do the comparison on equivalent cost basis, i.e. "what is the walltime difference if I run on a $60k all-CPU cluster versus a $60k GPU cluster". Or in terms of cloud compute cost / HPC allocation spend.
    - markstock 2y ago
      This has not been my experience in the academic/research side. Poison solver-based incompressible CFD regularly runs ~10x faster on equivalently-priced GPU systems, and has been doing so since I've been following it (since 2008). Some FFT-based solvers don't weak scale ideally, but that'd be even worse for CPU-based versions, as they use similar algorithms and would be spread over many more nodes.

That's the smallest of 4 experiments. It goes upto 140 million cells, where MI300X retains similar performance advantage of around 10% over Nvidia's H100.

by shihab 2y ago

That's the smallest of 4 experiments. It goes upto 140 million cells, where MI300X retains similar performance advantage of around 10% over Nvidia's H100.

semi-extrinsic 2y ago

Yeah, but the lack of comparison to a CPU-only system is suspicious. (As is the exaggerated plot without ticks and labels on the y-axis.) I've never seen a non-reactive incompressible flow simulation get substantial speedup on GPUs. There are well understood fundamental reasons why this is the case.

shihab 2y ago
I helped develop a hydro solver coupled with radiation at Los Alamos this summer. We observed from 7x upto 15x performance improvement on a single GPU compared to 64-core intel CPU. [1] Now granted, the flops to byte ratio for this program might be better than an avg fluid simulator. Also, our performance tanked when we moved to multi-node system. But I am aware of underlying reasons behind the scalibility issues and they don't feel like problems that can't be overcome. [1] https://github.com/lanl/HARD/tree/main https://github.com/lanl/HARD/tree/main
- semi-extrinsic 2y ago
  Nice, interesting solver! But yes, compressible/density-based solvers work a lot better on GPUs than incompressible/pressure-based solvers. When 80% of the runtime is spent solving a Poisson equation, GPU is not worth it. Especially if you do the comparison on equivalent cost basis, i.e. "what is the walltime difference if I run on a $60k all-CPU cluster versus a $60k GPU cluster". Or in terms of cloud compute cost / HPC allocation spend.
  - markstock 2y ago
    This has not been my experience in the academic/research side. Poison solver-based incompressible CFD regularly runs ~10x faster on equivalently-priced GPU systems, and has been doing so since I've been following it (since 2008). Some FFT-based solvers don't weak scale ideally, but that'd be even worse for CPU-based versions, as they use similar algorithms and would be spread over many more nodes.