The nice thing is that every gpu is like a little distributed-memory machine, and programming it for optimal memory locality (registers, shared, global) and ban

The nice thing is that every gpu is like a little distributed-memory machine, and programming it for optimal memory locality (registers, shared, global) and bandwidth is an essential skill. One step up is using multi-node distributed memory machines, which you could set up at home with openmpi, but is often easier to get access in an academic setting. In short, learn how to use CUDA and MPI. That's my personal take on "HPC", though obviously it can mean different things to different people.