4 ms·
How Machines See the Web: Exploring the Web Algorithmically
- Animats 11y agoIt's encouraging that someone is still thinking about how to make search better. There's surprisingly little work going on in that area any more. Vast amounts of effort are going into search engine optimization. Even trying to find any work on search engine improvement is difficult, because search results for that subject are buried under "search engine optimization" content. There's a "Search Quality Alliance".[1] That seemed promising until I found out what it was. [1] http://searchqualityalliance.org/ http://searchqualityalliance.org/
- doppenhe 11y agoI don't think we can really say there is little going on in search when Google and Bing both have 100s if not 1000s of engineers working on it. That said I do agree that I would expect a lot more ML and NLP tools to start showing up as part of search for helping to classify content (I am sure they do this internally just hasn't been user exposed). We built this demo because we thought there was no easy way to classify content sites (a problem we had been discussing with KDNuggets) and got inspired by some of the algos our community had added.
- wslh 11y agoThis research topic is called focused crawling. You can search more about this on https://scholar.google.com/scholar?q=focused+crawling&btnG=&hl=en&as_sdt=0%2C5&scilib=1&scioq=focused+web+crawler https://scholar.google.com/scholar?q=focused+crawling&btnG=&...