The usefulness of a computer isn't linear with speed. Usefulness is application-specific, of course, but in most applications usefulness is somewhere between lo

The usefulness of a computer isn't linear with speed. Usefulness is application-specific, of course, but in most applications usefulness is somewhere between log(speed) and sqrt(speed). For example: If you're solving NP problems (common in optimization), the size of the problem you can handle in a given amount of time is proportional to log(speed). If you're adding features to software, then any given feature requires some amount of compute power. Experience shows these to be power-law distributed. Early software versions started with the cheapest features, so now we're adding things on the right side of the distribution. So each new feature is likely to require additional compute power increasing like a power law. For a concrete example: when OSX added transparency and shadows to windows, it required every pixel in a window to be operated on twice. A doubling of a major component of the screen update time for a feature which only slightly increases user happiness/productivity. It had to wait until GPUs were fast enough to support it. In machine learning, the accuracy of results increases much less than linearly, somewhere between log(# parameters) and sqrt(# parameters). And the inference time is linear in the number of parameters. So it's a good thing that compute power is increasing exponentially or else progress would be really slow.