5 ms·
The challenge is not building the search engine, it’s in building the index. That’s why google wants every drop of data
by albertgoeswoof 8y ago
The challenge is not building the search engine, it’s in building the index.
That’s why google wants every drop of data
- laumars 8y agoTrue, but you have to start somewhere. Plus he did say his end goal was still 10 years away so it's not like he is he's under any disillusions about the work involved
- 0xcde4c3db 8y agoI've sometimes wondered whether it would work for a search engine to reduce the indexing problem by focusing more on quality than quantity. Rather than indexing everything in the universe and then trying to rank it, focus on maximizing ROI and keeping the aggregate quality of the corpus up by aggressively pruning low-quality paths up-front. In practice this might require splitting the difference between classic Yahoo and modern search engines, with manual maintenance of various black/white/greylists and rules to assign different quality metrics for different users on social media sites, which might reduce the effectiveness of this approach. Anyone know if something like this has been tried?
- actuallyalys 8y agoYou can sort of simulate this by searching discussion sites like Hacker News or Reddit. There's no pruning, but users do vote on what's most interesting or relevant to them. I find searching HN is useful when I'm looking for tools designed in the way HN readers tend to like: command-line, open source, and using a standard format.