Doesn't sound apocryphal at all. A well-written piece of custom code for a single multi-core CPU can easily run circles around an algorithm shoehorned into a u

Doesn't sound apocryphal at all. A well-written piece of custom code for a single multi-core CPU can easily run circles around an algorithm shoehorned into a useful but somewhat generic multiprocessor framework where the interconnect between the processors is the real culprit making the computation communication-limited. A related specific example would be molecular dynamics on GPUs versus CPUs. Not only are GPUs faster, they're faster than CPUs can be at this time because of communication issues. As in you can throw as many CPUs and servers as you want at the problem and they'll never beat a single $600 consumer GPU with its 120+ GB/s internal bus (note the absence of hyperbolic claims of 100x to 1000x faster, just faster). Or in other words, strong-scaling tasks don't work out so well on a weak-scaling architecture.