A lot of the ways programmers write slow code ends up with multiplicative factors you can't practically recover with simply throwing more hardware at it, becaus

A lot of the ways programmers write slow code ends up with multiplicative factors you can't practically recover with simply throwing more hardware at it, because it's really easy for us to write something 100x or 1000x slower (even in the same big-O class) by using a slow language with poor cache locality, that makes it easy to do more work than was necessary, by poorly using threading resources, etc. There's a lot of multiplicative factors that can stack together fairly quickly. On the plus side, it turns out recovering most of that isn't that hard with a bit of careful language selection and just a bit of care in writing code. I think this is one of the somewhat subtle reasons for Go's success is that it trims away most of the slow things that a lot of "scripting" languages do without throwing away too much of the power, and I anticipate the continued entry and success of other languages like Nim into this space on the language chart in the next few years. To get to screaming fast may take a lot of effort, but "pretty fast, let's parallelize what we have instead of hyperoptimizing it" isn't too hard. Because of all of these multiplicative slowdowns, even at very small scales it's still generally a better idea to start by optimizing and removing some of your multiplicative slowdowns before you just throw more hardware at it. Compute is merely cheap, not free. It's the rare bit of code that nobody's spent any time optimizing, but is also free of these multiplicative slowdown problems.