Edit: there appears to be such a project, at least on the crawling side: http://www.commoncrawl.org/ http://www.commoncrawl.org/ I'd say there's a combination

Edit: there appears to be such a project, at least on the crawling side: http://www.commoncrawl.org/ http://www.commoncrawl.org/ I'd say there's a combination of factors, the first (and most important) being that Google is good enough for most people. You'd need to coordinate crawling so as not to turn it into a giant DDoS machine; speed will be an issue due to geo distribution, variable hardware and result sets. Validity and reliability of the data would also be issues, and would probably require several peers to "agree" to consistency, but in a way that does not allow easy gaming of results. I suppose they're all solvable, though I think there would have to be a powerful incentive to do so. I imagine it'd be quite pricey too for the individual, though perhaps Gabriel Weinberg* could weigh in there. [*] http://www.gabrielweinberg.com/ http://www.gabrielweinberg.com/