3 ms·

You fail to understand that crawling the web is 0.5% of the picture. Google has the best treatment of that data to serve ads, personalise experience and that's

You fail to understand that crawling the web is 0.5% of the picture. Google has the best treatment of that data to serve ads, personalise experience and that's what their value is. You think making a bit crawler would nullify google, but you'd have to do the other 99.95% to be better (which costs money most people don't have).

mindcrime 8y ago
You think making a bit crawler would nullify google No, I don't. I didn't say anything even remotely like that. What I'm saying is that crawling is a prerequisite for all of the "... best treatment of that data to serve ads, personalise experience ..." stuff. That's why this topic matters: because if people aren't allowed to crawl (or can't do it effectively for technical reasons, but that's a subtly different issue), then they can't do anything else. Just to be clear, I am most emphatically not saying that just building a crawler is enough to compete with Google. What I'm saying is that you can't build a competitive search engine without that "0.05%" bit because it's required to enable the other 99.95%.