3 ms·
Are there any "open" search index initiatives going on? Like a curated wikipedia/openstreemap of internet search indexes (with ignore-lists like uBlock has for
by riggsdk 5y ago
Are there any "open" search index initiatives going on?
Like a curated wikipedia/openstreemap of internet search indexes (with ignore-lists like uBlock has for ads)?
I fully realize the impossibility of a handmade search index for the entire web, the internet is just too impossibly large and ever changing. I'm more thinking of subsets of various topics of interest. Different communities could pop up that made specialized crawling algorithms for very specific topics and you could subscribe to those indexes for a very personalized search experience. A mix of automatic classification, indexing combined with hand-tweaked tagging and weeding out spam.
Over time you could have a pretty "broad" search engine - kinda like how OpenStreetMap was initially terrible (coverage wise) but is today extremely competitive with the big guys.
With it I could subscribe to or possibly even aggregate/proxy other indexes into a personalized search system. That way I could make a super specialized search that only searched indexes made for information search on technical topics and have another index explicitly filter out unwanted spammy websites.
Some "pre-index" on the search front-end could intelligently decide which indexes to query based on your search string. If you are dissatisfied with the indexes it chose to query you could manually tweak it and redo the search.
At the very least it would be awesome if search engines had a way to submit an "ignore-list" - an index over known crap/spammy websites. It's not feasible for me to keep adding specific ignored urls to a search term. When I search for images online I would prefer to ignore Pinterest by default.
It really irks me that when searching for information that Google defaults to thinking I'm looking for a product to buy. Sometimes I would just prefer to exclude ALL websites that generally only acts as a retail/webshop for goods. That fine grained search method is just not possible today.