4 ms·

DDG and even Bing/Yahoo both need to index much more of the web if they want to be competitive. You just can't beat anyone in quality when you have fewer result

DDG and even Bing/Yahoo both need to index much more of the web if they want to be competitive. You just can't beat anyone in quality when you have fewer results to work with - it's just simple math. I hope DDG uses some of its investment money on solving this problem (as well as speed, but speed is secondary). The bang codes, privacy, etc are all great.

deleted 15y ago
[deleted]
mikkom 15y ago
isn't ddg a meta search engine (if I remember correctly they scraped google and used yahoo search api)? Or has this changed?
- crazedpsyc 15y ago
  It uses Yahoo, Bing, and a number of other sources (but not google). See http://help.duckduckgo.com/customer/portal/articles/216399-sources http://help.duckduckgo.com/customer/portal/articles/216399-s... :)
  - AznHisoka 15y ago
    Then they are limited to what's in Bing's/Yahoo's index, and their index is vastly inferior to Google's... it's not even close. This is partially why Google has better results - they simply crawl more of the web (plus their algorithm too, of course). Why can't DDG just cut their losses and invest in building their own index? It's much more worth it in the long run. All DDG is doing is adding cool, nice to have UI features on top of Bing/Yahoo, but what about the core?
    - crazedpsyc 15y ago
      Well for one, results come from far more sources than /just/ bing and yahoo, and they are also put through a custom algorithm. Then there is the 0-click info, official sites, etc. As for building an index, I'm not sure what the plans are there, however there has been some discussion of a distributed spider. Obviously, spidering would mean a LOT of resources directed away from search (don't forget the limited human resources too).
    - mikkom 15y ago
      I'm quite sure they used to use google in the past > Why can't DDG just cut their losses and invest in building their own index? That's far from trivial task.. Scraping and using other peoples results is easy, building and scaling a huge database like search index is one of the hardest problems you can find.
      - AznHisoka 15y ago
        Ok that's fair. But in that case, I don't understand what compelling way DDG is differentiating themselves from other search engines besides a few cool, nice, thin value features.
        mikkom 15y ago
        .. that's almost exactly what I'm thinking too ..
    - minwcnt5 15y ago
      DDG's main schtick is that they don't track users. It would be nearly impossible to match Google or Bing's ranking relevance without using signals derived from logged user traffic. Their core principles essentially ensure they won't ever be a serious competitor to the leading search engines, unless a lot of people decide that they're willing to sacrifice a lot of relevance in their search results in exchange for stricter privacy.
    - rieter 15y ago
      Because right now there are only 2 companies in the world with sufficient technical chops to build a competent end-to-end modern search engine. It's a HARD problem. No startup can do it.
wisty 15y ago
> it's just simple math More results doesn't always give better results. Generally, they do, but depending on the problem it's often very marginal. The actual ranking algorithm is more important, especially (I suspect) pre-processing. Google's main advantage (I'd guess) is its expertise in algorithms and processing.

deleted 15y ago

[deleted]

mikkom 15y ago

isn't ddg a meta search engine (if I remember correctly they scraped google and used yahoo search api)? Or has this changed?

crazedpsyc 15y ago
It uses Yahoo, Bing, and a number of other sources (but not google). See http://help.duckduckgo.com/customer/portal/articles/216399-sources http://help.duckduckgo.com/customer/portal/articles/216399-s... :)
- AznHisoka 15y ago
  Then they are limited to what's in Bing's/Yahoo's index, and their index is vastly inferior to Google's... it's not even close. This is partially why Google has better results - they simply crawl more of the web (plus their algorithm too, of course). Why can't DDG just cut their losses and invest in building their own index? It's much more worth it in the long run. All DDG is doing is adding cool, nice to have UI features on top of Bing/Yahoo, but what about the core?
  - crazedpsyc 15y ago
    Well for one, results come from far more sources than /just/ bing and yahoo, and they are also put through a custom algorithm. Then there is the 0-click info, official sites, etc. As for building an index, I'm not sure what the plans are there, however there has been some discussion of a distributed spider. Obviously, spidering would mean a LOT of resources directed away from search (don't forget the limited human resources too).
  - mikkom 15y ago
    I'm quite sure they used to use google in the past > Why can't DDG just cut their losses and invest in building their own index? That's far from trivial task.. Scraping and using other peoples results is easy, building and scaling a huge database like search index is one of the hardest problems you can find.
    - AznHisoka 15y ago
      Ok that's fair. But in that case, I don't understand what compelling way DDG is differentiating themselves from other search engines besides a few cool, nice, thin value features.
      - mikkom 15y ago
        .. that's almost exactly what I'm thinking too ..
  - minwcnt5 15y ago
    DDG's main schtick is that they don't track users. It would be nearly impossible to match Google or Bing's ranking relevance without using signals derived from logged user traffic. Their core principles essentially ensure they won't ever be a serious competitor to the leading search engines, unless a lot of people decide that they're willing to sacrifice a lot of relevance in their search results in exchange for stricter privacy.
  - rieter 15y ago
    Because right now there are only 2 companies in the world with sufficient technical chops to build a competent end-to-end modern search engine. It's a HARD problem. No startup can do it.

wisty 15y ago

> it's just simple math More results doesn't always give better results. Generally, they do, but depending on the problem it's often very marginal. The actual ranking algorithm is more important, especially (I suspect) pre-processing. Google's main advantage (I'd guess) is its expertise in algorithms and processing.