Having dealt with applications spanning thousands of machines, I'd say there will always be value to optimizing programs on a single machine. The prerequisite

Having dealt with applications spanning thousands of machines, I'd say there will always be value to optimizing programs on a single machine. The prerequisite to distributed computing is being able to program a single computer. If you can save an order of magnitude in machines, you will always avoid some scaling problem. 10 machines, 100 machines, and 1000 machines all have different problems.