There's a lot of ways to program these GPU supercomputers, and you are maybe overestimating how much of the HPC code is written in CUDA. Some is, for sure, but

There's a lot of ways to program these GPU supercomputers, and you are maybe overestimating how much of the HPC code is written in CUDA. Some is, for sure, but there's also a lot using OpenACC or OpenMP (C, C++, or Fortran with annotations), or frameworks like Kokkos. Recently, it also became possible to use C++17 stdpar and get GPU acceleration (https://docs.nvidia.com/hpc-sdk/compilers/c++-parallel-algorithms/index.html https://docs.nvidia.com/hpc-sdk/compilers/c++-parallel-algor...).