Chatting about this with you made me challenge my knowledge of how web search works. Many articles I've found suggest that the hardest problem to solve isn't co

Chatting about this with you made me challenge my knowledge of how web search works. Many articles I've found suggest that the hardest problem to solve isn't competing on the size of Google but the relevance of Google. Which is partly to do with the size and power of their datacenters. But also to do with smart engineering of the servers, database, and algorithms. The algorithms not only determine the how to order the results but which pages to crawl and how often. What this all makes me think of is all the work being put into AI research lately. IBM, Google, Facebook, Baidu etc. There is a race to AI and the winner could very well make a better Google.