You are underestimating what is necessary to do implement tractable algorithms on big dataset. The classic index/search problem would be extrmely simple if it

You are underestimating what is necessary to do implement tractable algorithms on big dataset. The classic index/search problem would be extrmely simple if it weren't for the big data involved: it is basically hash table lookups plus set intersection, and you're done. Doing the slightest bit more is disproportionally harder. Improving relevance is probably best viewed as a recommender problem, and just look at the Netflix prize how hard it can be to squeeze the last bits of performance out of their (toy!) dataset.