5 ms·
Do people use Cilk heavily in the high-performance computing world? They did not when I was part of it. I think OpenMP (https://www.openmp.org/ https://www.open
by scott_s 7y ago
Do people use Cilk heavily in the high-performance computing world? They did not when I was part of it. I think OpenMP (https://www.openmp.org/ https://www.openmp.org/) is what more people would reach for. It started as a way to parallelize loops, but has had a task framework for a long time which I believe also allows composeable parallelism.
OpenMP, however, depends on not just compiler extensions, but pragmas, so it tends to not find traction outside of the HPC world.
- krapht 7y agoThe real gain is with MPI, anyway... shared memory is a trap. All the projects I've worked on have exploited parallelism through message passing, or a battle tested multithreaded linear algebra library.
- BubRoss 7y agoThere are many techniques and calling any of them the one solution and all others a trap is extremely myopic. Shared memory is extremely useful for IPC and fork join parallelism.
- vmchale 7y ago> All the projects I've worked on have exploited parallelism through message passing What? Lots of parallelism doesn't use message passing!
- seanmcdirmid 7y agoGPU is the way to go these days to get most parallelism, and it is definitely the shared memory variety. Message passing is the go to when parallelism needs to cross machine boundaries.
- steev 7y agoGPU parallelism is highly specialized and requires very specific workloads (e.g., many linear algebra workloads). GPUs are a bad choice for general parallelized tasks.
- seanmcdirmid 7y agoYes, but for heavy numeric problems, GPUs are your best bet if you can make them work. The question is what is Julia being used for, I guess.
- scott_s 7y agoA quibble: it's not necessarily "heavy numeric" problems, but problems where the calculations involve extensive data-reuse. (Of which matrix operations are the most obvious.) The GPU is massively parallel, but the up-front cost of transferring data to the GPU is quite high. Hence, in order for that up-front cost to be worth it, the algorithms run on the GPU need to re-use that data many times. If the algorithms running on the GPU don't make extensive reuse of the data sent to it, it would be faster to just do the calculation on the host CPU.
- marmaduke 7y agoMPI is the HPC horizontal scaling strategy, for when you need to go from 1 core to 500k.
- gpderetta 7y agoMy understanding is that, at least in c/Fortran/c++ land common practice is openmp within a machine and MPI across machines. I've done some work of the sort but I'm far from an expert though.
- steev 7y agoYou are correct.
- minxomat 7y agoCilk maybe not, but TBB is just a library (or more accurately a toolbox of primitives), which integrates nicely with OpenMP: https://software.intel.com/en-us/node/506128 https://software.intel.com/en-us/node/506128
- gpderetta 7y agoWhile extremely elegant and minimalistic, I do not think cilk ever gained much traction. GCC has deprecated it and is removing it. I think that OpenMP was too entrenched in the language extension space (and has since grown at least the potential of cilk like functionality), while more portable libraries (TBB for example) covered everything else. I think it has been highly influential though.
- scott_s 7y agoYes, Cilk has been enormously influential. I have personally cited it many times. But I always had the impression it was a little too early for its time, and was never picked up by the HPC community, hence my question.