Last time I did something in C requiring spreading work across a cluster of machines it was with MPI(CH). Docs available at mpich.org. This was Monte Carlo simu

Last time I did something in C requiring spreading work across a cluster of machines it was with MPI(CH). Docs available at mpich.org. This was Monte Carlo simulation for hyper-dimensional (~8 IIRC) asset allocation and thus the simulations were easily divisible—recombining was just simple arithmetic—and the network I/O was minimal. Interesting tidbit (to me anyway): This was 2006, before you could use something like AWS for the purpose and we were trying to keep costs to a minimum. (IBM had their "public one" grid but it was unusable.) The Core 2 Duo processor had just come out so I hired an intern from Cal—now a PhD and brilliant engineer at Netflix—to figure out the optimal overclocking rig and we built 32 chassis consisting of just motherboard, RAM, NIC, custom cooling/heatsink, and power supply. The problem was then how to deploy these in a colo. At the time there were some low end providers at 200 Paul willing to get creative with a cabinet so I found a machinist (metal worker?) able to cut some custom aluminum shelving on top of which we could stack the ATX cases. Rigged the boxes up to network boot off one of the nodes, compiled our application with Intel's C++ compiler to take advantage of the SIMD/SSE3 instruction set, and away we went running billions of simulations on a startup budget.