So HPC means like ten different things. For example another commentor mentions low-latency concerns in finance, and that's something I have zero experience wit

So HPC means like ten different things. For example another commentor mentions low-latency concerns in finance, and that's something I have zero experience with. HPC has often also meant writing a lot of C++ to do e.g. MD or something. These days, I consider myself HPC-adjacent -- I write scientific ML software, often for use on pretty beefy hardware (TPU pods etc.) So at least for that, here's an off-the-cuff list of a few items that come to mind: - Know JAX. Really, really well: its internals, how its transforms work. It's definitely a bit bumpy in places, but it's still one of the best things we have for easily scaling programs, e.g. through `jax.pmap`, being able to test on CPU and then run on TPU, etc. - Triton! New(-ish) kid on the block for GPU programming. - How CPUs work: L1/L2/L3 caches, branch prediction, etc. Parallelism via OpenMP. - How GPUs work: warps etc. - How BLAS works (e.g. tiling).. - Compiler theory. Inlining functions, argment aliasing, NRVO, ... - Know autodiff well. E.g. have a read of the Dex paper, and the concerns with doing autodiff through index operations. Modern scientific computing is moving towards a ubiquitously autodifferentiable future. - ... plus loads more, haha. Probably I'm still missing ten different things that another reader considers crucial.