3 ms·

Can someone describe to me a search engine that’s immune to such problems and also searches a large variety of old and new sites?

• majormajor 5y ago
  We often talk about this problem as if we expect Google to solve a social and economic people problem purely through math alone. The appeal of open-to-all search as a way of navigating the web was that there was a huge long tail of interesting stuff that would be hard to manually index and categorize. If the long tail of interesting stuff has fully drowned in a sea of spam and crap, I'm not sure that it still makes that much sense over something smaller but human-curated. Perhaps the trick would be human curation with extensive and always-evolving AI tools to speed up the curation. You have to get past the filter to get in, versus being in by default unless you're blatant enough to get banned. There is a layer of human judgement in addition to the algorithm's score of the content, and additionally that gives you an extra scoring factor on the algorithms yourself - the humans should be able to help direct the development of the algorithm to fight spam more preemptively. Would a mix of that give us a bigger internet than the entirely-manual "web directory" days of 1997 or so, but a less-shit-filled one than today's?