If you read the original Anatomy of a Search Engine paper by Google co-founders, you'd be amazed at the constraints they had to engineer around with back then.

If you read the original Anatomy of a Search Engine paper by Google co-founders, you'd be amazed at the constraints they had to engineer around with back then. Many of the constraints that they had to engineer around don't exist anymore. The landscape has changed significantly since 1998. The decrease in bandwidth costs, increase in computing power, RAM, infrastructure as a service, and the rapidly moving ML research that pops out new NLP techniques. You also have to consider Paul Graham's advice on working backwards. Google is nowhere near the final form of information retrieval. In an ideal world, I would have an answer to a search query instantaneously. Even we solve that problem, then you begin to try to solve the problem of why you even had to query for information in the first place.