3 ms·

I feel that a discussion of http://en.wikipedia.org/wiki/Amdahl's_law http://en.wikipedia.org/wiki/Amdahl's_law should be mandatory when introducing parallel pr

by platz 12y ago

I feel that a discussion of http://en.wikipedia.org/wiki/Amdahl's_law http://en.wikipedia.org/wiki/Amdahl's_law should be mandatory when introducing parallel programming Small sequential portions of an otherwise parallel algorithm can have huge effects on the overall running when trying to scale up. "parconc" explains this while discussing a parallel version of k-means, talks about how things like granularity of data needs to be fine-tuned for parallel algos, and provides some nice visualizations into what the CPU's are actually doing on a timeline: http://chimera.labs.oreilly.com/books/1230000000929/ch03.html#sec_par-kmeans-perf http://chimera.labs.oreilly.com/books/1230000000929/ch03.htm... Overall I think multicore is a good tool to have in your toolbox, but it seems like there needs to be a lot of tuning and effort to get good rewards for the time invested.

flatline 12y ago
This is certainly true for some applications, but originally when Amdahl's law was formulated, people made estimates based on e.g. 95% of an application being parallelizable, and thus fairly rapidly reaching a point of diminishing returns. In practice, however, there's often a simple reformulation of the problem that can result in a much higher percentage, and Amdahl's becomes less an obstacle than it would first appear. There are still many problems that scale more or less linearly with the number of processing units and are "trivially parallelizable".