5 ms·

Yep. A lot of folks appear to be pretty confused about retail search engines verses the actual index -- Google, Bing, and (at lower quality) Yandex are the only

Yep. A lot of folks appear to be pretty confused about retail search engines verses the actual index -- Google, Bing, and (at lower quality) Yandex are the only viable English-language search indices right now. Almost every search box you find on the Internets is powered by one of the three.

abtinf 7y ago
> Google, Bing, and (at lower quality) Yandex are the only And DuckDuckGo.
- kick 7y ago
  DuckDuckGo is just a metacrawler. Depending on the search, it uses either Yandex or Bing.
  - wolco 7y ago
    Doesn't it maintain it's own index on certain terms based on user actions/feedback that changes the bing results resulted? I thought there was an improvement feedback loop.
    - kick 7y ago
      That doesn't seem to be the case; the only reason they use r.duckduckgo.com is for redirecting without the REFERER according to their docs, so there doesn't seem to be a way for them to do that.
  - codesections 7y ago
    According to DuckDuckGo, > To do that, DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google). Do you view that as misleading/incorrect?
    - jhoechtl 7y ago
      Yes, as we know nothing about the percentage of traffic which is coming from google, bing or any of the remaining 398 ressources. I can only assume the contribution is abysmal, given the almight of the one trully.
    - wongarsu 7y ago
      I read that as "Over four hundred sources for instant answers (the big box at the top) etc. What you usually think of as search results ("traditional links") comes from multiple partners, mostly from Bing (and never from Google, which only leaves Yandex and niche providers)". That fits well with what GP claimed.
- plapetomain 7y ago
  No. Ddg gets most of its index from bing.
pythux 7y ago
Did You check Cliqz already? Independent search index: https://beta.cliqz.com/ https://beta.cliqz.com/ There is a lot of technical details about how it’s built on the tech blog: https://0x65.dev/ https://0x65.dev/ (Disclaimer: I work at Cliqz)
- pmontra 7y ago
  I've been using your service as my main search engine on desktop and phone for more than one month (I made an xml file to load it as search engine.) It's okish but not very good at dealing with misspelling, with news and with images. Sometimes I touch the Google button at the end of the page to find what I'm looking for. I hope you keep track of the keywords that go to Google and use them to improve your index. Anyway, it's a good service, thank you.
  - __ka 7y ago
    Hey, thanks for using Cliqz. We're working hard on improvements for each one of the things you mentioned (in that order as well) - so things will definitely get much smoother over the next couple of months.
- busymom0 7y ago
  Maybe unrelated but what are the server costs and hardware used etc for such a thing?
- jefftk 7y ago
  I looked at Cliqz a few months ago, and the crawl didn't seem competitive with Google or Bing: https://news.ycombinator.com/item?id=21676600 https://news.ycombinator.com/item?id=21676600
  - jhoechtl 7y ago
    You mean they are not as competitive as a billion dollar company 20 years in business?
    - 7y ago
dna_polymerase 7y ago
Qwant has their own index (or atleast parts) IIRC.
tmpAa 7y ago
SwissCows has its own indexer and I believe they don't rely on any external search source like Bing. They should push for better marketing, less nationalism & better UI like duckduckgo and they would go pretty far imo, their results are often better than DDG edit : my bad, actually they do use Bing in addition to their own crawler
kristopolous 7y ago
Yacy. Legendary yacy. Still around. Also gigablast

abtinf 7y ago

> Google, Bing, and (at lower quality) Yandex are the only And DuckDuckGo.

kick 7y ago
DuckDuckGo is just a metacrawler. Depending on the search, it uses either Yandex or Bing.
- wolco 7y ago
  Doesn't it maintain it's own index on certain terms based on user actions/feedback that changes the bing results resulted? I thought there was an improvement feedback loop.
  - kick 7y ago
    That doesn't seem to be the case; the only reason they use r.duckduckgo.com is for redirecting without the REFERER according to their docs, so there doesn't seem to be a way for them to do that.
- codesections 7y ago
  According to DuckDuckGo, > To do that, DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google). Do you view that as misleading/incorrect?
  - jhoechtl 7y ago
    Yes, as we know nothing about the percentage of traffic which is coming from google, bing or any of the remaining 398 ressources. I can only assume the contribution is abysmal, given the almight of the one trully.
  - wongarsu 7y ago
    I read that as "Over four hundred sources for instant answers (the big box at the top) etc. What you usually think of as search results ("traditional links") comes from multiple partners, mostly from Bing (and never from Google, which only leaves Yandex and niche providers)". That fits well with what GP claimed.
plapetomain 7y ago
No. Ddg gets most of its index from bing.

pythux 7y ago

Did You check Cliqz already? Independent search index: https://beta.cliqz.com/ https://beta.cliqz.com/ There is a lot of technical details about how it’s built on the tech blog: https://0x65.dev/ https://0x65.dev/ (Disclaimer: I work at Cliqz)

pmontra 7y ago
I've been using your service as my main search engine on desktop and phone for more than one month (I made an xml file to load it as search engine.) It's okish but not very good at dealing with misspelling, with news and with images. Sometimes I touch the Google button at the end of the page to find what I'm looking for. I hope you keep track of the keywords that go to Google and use them to improve your index. Anyway, it's a good service, thank you.
- __ka 7y ago
  Hey, thanks for using Cliqz. We're working hard on improvements for each one of the things you mentioned (in that order as well) - so things will definitely get much smoother over the next couple of months.
busymom0 7y ago
Maybe unrelated but what are the server costs and hardware used etc for such a thing?
jefftk 7y ago
I looked at Cliqz a few months ago, and the crawl didn't seem competitive with Google or Bing: https://news.ycombinator.com/item?id=21676600 https://news.ycombinator.com/item?id=21676600
- jhoechtl 7y ago
  You mean they are not as competitive as a billion dollar company 20 years in business?
  - 7y ago

dna_polymerase 7y ago

Qwant has their own index (or atleast parts) IIRC.

tmpAa 7y ago

SwissCows has its own indexer and I believe they don't rely on any external search source like Bing. They should push for better marketing, less nationalism & better UI like duckduckgo and they would go pretty far imo, their results are often better than DDG edit : my bad, actually they do use Bing in addition to their own crawler

kristopolous 7y ago

Yacy. Legendary yacy. Still around. Also gigablast