3 ms·

I'd love a search engine with more querying power limited to a niche. But I doubt limiting topical scope would do much to keep computational complexity from blo

I'd love a search engine with more querying power limited to a niche. But I doubt limiting topical scope would do much to keep computational complexity from blowing up. A while back some startup was charging per search to work around that.

greglindahl 8y ago
blekko provided 2,000 niches and it wasn't enough to keep us alive. Also, you need a web-wide crawl to build a good niche engine.
tgb 8y ago
I wasn't really thinking a niche topic but a niche use-case. Google is bad at doing literal, verbatim string searches these days and all I want sometimes is like Google 2005ish era search, just PageRank based and no modern machine learning, etc. You can have it simpler than Google did at the time since SEO against your niche engine is unlikely, so tech wise you might be more like 1998 Google. The main bottleneck seems to be what others here are suggesting: there's just too much web to index these days unless you're Bing-scale or bigger.
- auganov 8y ago
  Well that's sort of what I'm thinking of too. I'm talking about the idea that this might be doable by limiting topical scope. So for example you'd only index some portion of tech related sites.
- wutbrodo 8y ago
  > Google is bad at doing literal, verbatim string searches these days and all I want sometimes is like Google 2005ish era search, just PageRank based and no modern machine learning, etc You can get most of the way to this with the verbatim option, but I think they make it difficult to make the default. > By using the Verbatim tool Google will not make the following changes: • Personalizing your search using websites you have visited before; • Including synonyms of your search terms; • Automatic spelling corrections; • Searching for words with the same stem e.g. “Shopping” when searched for “shop”; • Finding results that match similar terms to those in your query.

greglindahl 8y ago

blekko provided 2,000 niches and it wasn't enough to keep us alive. Also, you need a web-wide crawl to build a good niche engine.

tgb 8y ago

I wasn't really thinking a niche topic but a niche use-case. Google is bad at doing literal, verbatim string searches these days and all I want sometimes is like Google 2005ish era search, just PageRank based and no modern machine learning, etc. You can have it simpler than Google did at the time since SEO against your niche engine is unlikely, so tech wise you might be more like 1998 Google. The main bottleneck seems to be what others here are suggesting: there's just too much web to index these days unless you're Bing-scale or bigger.

auganov 8y ago
Well that's sort of what I'm thinking of too. I'm talking about the idea that this might be doable by limiting topical scope. So for example you'd only index some portion of tech related sites.
wutbrodo 8y ago
> Google is bad at doing literal, verbatim string searches these days and all I want sometimes is like Google 2005ish era search, just PageRank based and no modern machine learning, etc You can get most of the way to this with the verbatim option, but I think they make it difficult to make the default. > By using the Verbatim tool Google will not make the following changes: • Personalizing your search using websites you have visited before; • Including synonyms of your search terms; • Automatic spelling corrections; • Searching for words with the same stem e.g. “Shopping” when searched for “shop”; • Finding results that match similar terms to those in your query.