5 ms·

What is it like to program for a supercomputer? Where can I learn about it? Not having an academic affiliation, is there anywhere I can run small jobs affordabl

What is it like to program for a supercomputer? Where can I learn about it? Not having an academic affiliation, is there anywhere I can run small jobs affordably?

javierluraschi 7y ago
Programming a supercomputer feel like having to write a program that uses thousands of computers, that’s about it. Some supercomputer vendors have specialized software running, but others also support open source distributed computing frameworks like Apache Spark. If you are wondering why we call them supercomputers, it’s because it’s not as easy to just buy thousands of computers. Even easy things like shipping, installation and power consumption becomes a problem on its own. So rather than buying thousands of machines it pays off to buy them through a supercomputer vendor that will optimize space, power consumption, install cluster management software, etc. So if you want to get started, I would just get a small n-node cluster from any cloud provider and get better at writing distributed code.
- closeparen 7y ago
  I deploy to hundreds of machines at work, but that is just load balanced RPC and message queue processing workloads, interacting with other load balanced RPC and message queue workloads. The little information I can make sense of about “HPC” does not seem familiar through that lens.
  - tenebrisalietum 7y ago
    Based on a very shallow reading of Infiniband, I think some of the interconnects of supercomputers are essentially DMA-based rather than network-based. So it's like writing something massively multithreaded (your barrier is one step above shared memory) versus multihosted. I could be completely 100% wrong.
shaklee3 7y ago
Mostly learn how to use MPI.
jabl 7y ago
MPI is by far the dominant method for communicating among nodes. So learn that. MPI works fine on a multicore machine as well (launch one MPI rank per core), so you can run on your own machine, no need to use the cloud or anything like that. For using GPU's, there's CUDA, or offloading with OpenMP or OpenACC.
- martinpw 7y ago
  Relevant given the article topic - Cray is ending OpenACC support: https://twitter.com/hpc_guru/status/1113565651350589441 https://twitter.com/hpc_guru/status/1113565651350589441
  - jabl 7y ago
    Indeed, OpenMP offload and OpenACC seem so close to each other (Disclaimer, I haven't used either) that it's probably better if the world would converge on one of them.

What is it like to program for a supercomputer? Where can I learn about it? Not having an academic affiliation, is there anywhere I can run small jobs affordably?

javierluraschi 7y ago

Programming a supercomputer feel like having to write a program that uses thousands of computers, that’s about it. Some supercomputer vendors have specialized software running, but others also support open source distributed computing frameworks like Apache Spark. If you are wondering why we call them supercomputers, it’s because it’s not as easy to just buy thousands of computers. Even easy things like shipping, installation and power consumption becomes a problem on its own. So rather than buying thousands of machines it pays off to buy them through a supercomputer vendor that will optimize space, power consumption, install cluster management software, etc. So if you want to get started, I would just get a small n-node cluster from any cloud provider and get better at writing distributed code.

closeparen 7y ago
I deploy to hundreds of machines at work, but that is just load balanced RPC and message queue processing workloads, interacting with other load balanced RPC and message queue workloads. The little information I can make sense of about “HPC” does not seem familiar through that lens.
- tenebrisalietum 7y ago
  Based on a very shallow reading of Infiniband, I think some of the interconnects of supercomputers are essentially DMA-based rather than network-based. So it's like writing something massively multithreaded (your barrier is one step above shared memory) versus multihosted. I could be completely 100% wrong.

shaklee3 7y ago

Mostly learn how to use MPI.

jabl 7y ago

MPI is by far the dominant method for communicating among nodes. So learn that. MPI works fine on a multicore machine as well (launch one MPI rank per core), so you can run on your own machine, no need to use the cloud or anything like that. For using GPU's, there's CUDA, or offloading with OpenMP or OpenACC.

martinpw 7y ago
Relevant given the article topic - Cray is ending OpenACC support: https://twitter.com/hpc_guru/status/1113565651350589441 https://twitter.com/hpc_guru/status/1113565651350589441
- jabl 7y ago
  Indeed, OpenMP offload and OpenACC seem so close to each other (Disclaimer, I haven't used either) that it's probably better if the world would converge on one of them.