But at least today, searching isn't just a matter of a better web crawl, or a better algorithm against web data. A key component -- perhaps the key component --

But at least today, searching isn't just a matter of a better web crawl, or a better algorithm against web data. A key component -- perhaps the key component -- of providing good search results is being able to incorporate user feedback from previous searches. Imagine a company with a 100% market share. Your startup wants to complete. It crawls the web, applies the latest in NLP and semantic search technology and produces an index. And, of course, buys a Super Bowl ad. The public is curious; they try your product once. And they discover that it sucks, so they never come back. You simply can't compete on relevance without the user feedback. But without the volume of users, you'll never get the feedback you need. This is almost certainly why Microsoft made that crazy large offer for Yahoo all those years ago; it was their attempt to acquire volume to improve their relevance. Consider the "Bing Challenge". I haven't tried it personally, but the feedback on HN has generally been "meh". And yet Bing claims that in blind tests, some better-than-expected percentage of users prefer the Bing results. How can this be? Well, one explanation is that the test has been cooked somehow to prefer Bing. But another explanation is that Bing's existing users are more likely to be "average users", while readers of HN are "techies" who overwhelmingly prefer Google. So Bing may simply be producing results that are preferred by the kind of customers who are already using Bing -- and against search terms which are more likely to be used by the kind of customers who are already using Bing. This is consistent with the hypothesis that user feedback is the primary driver of search quality. Disclaimer: I work at Microsoft, but not at Bing. And it's a job, not a meaningful personal relationship.