I should clarify -- initially I/O bound, but with pipelining, we are able to "impedance match" the flow of data coming off the backing store with the GPU's comp

I should clarify -- initially I/O bound, but with pipelining, we are able to "impedance match" the flow of data coming off the backing store with the GPU's computational progress. I suppose the better word is "I/O heavy," rather than "I/O bound." We're certainly not the first to describe mapreduce as "embarrassingly parallel," but I can defend myself nonetheless: A shuffle executes in parallel where each unit re-indexes into another unit. Reduction is parallelized over each key, and even finer granularity can be achieved by considering a tree-based approach: http://developer.download.nvidia.com/compute/cuda/1.1-Beta/x86_website/projects/reduction/doc/reduction.pdf http://developer.download.nvidia.com/compute/cuda/1.1-Beta/x... In fact, I used this tree reduction approach to merge non-disjoint clusters of friend groups in my social network clustering algorithm. Parallelizing the input parsing is problem-specific, but even that is possible in many cases.