7 ms·

DuckDuckGo Founding Member in Global Privacy Control Standards Effort

143 points by tagawa 6y ago

TekMol 6y ago
If Bing ever drops DuckDuckGo, will there be a search engine they can fall back on?
- 02020202 6y ago
  they use bing only for some stuff, not as their backend. they will do just fine without bing.
  - ffpip 6y ago
    Do you have any source? They have always avoided the question of how much they were dependent on Bing.
    - notRobot 6y ago
      AFAICT, other than those information boxes that turn up on the top of searches, the actual search results are almost entirely powerer by bing, although as you said, they've always dodged the question.
    - mrweasel 6y ago
      No one has a source for “DuckDuckGo is just Bing” either. We know that they use Bing, but not for what, people have just asserted that DDG is just Bing (or Yandex), but no one seems to be able to provide at source to back up the claim.
      - pb7 6y ago
        >To do that, DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).[0] Seems pretty clear their standard search results come from Bing. It even used to say "multiple partners like Bing, Yahoo, etc" but Yahoo was also powered by Bing. They like to change the wording to make themselves sound less reliant on Bing even though if Bing were to cut off access, DuckDuckGo would immediately cease working. [0] https://help.duckduckgo.com/duckduckgo-help-pages/results/sources/ https://help.duckduckgo.com/duckduckgo-help-pages/results/so...
        mrweasel 6y ago
        I’ve read that multiple times, and no, that’s not clear. At best we can read it as “Results from Bing are mixed into their search results”. Most likely a high percentage, but we don’t know.
        pb7 6y ago
        Mission accomplished then because they don't want to make it clear. If you really care that much, go back through all the different revisions of that statement through the Internet Archive and see how many times they've been dancing around saying the exact same thing: it's just fucking Bing. Their own crawler returns results for very specific limited topics in their segregated boxes. If they had even one source for one specific query, say "purple cats" that they didn't use Bing for, they could technically claim it's only "mostly" Bing.
      - sct202 6y ago
        We can infer some things about DDG based published data on crawler and bot activity. DDG's index is pretty comparable to both Google and Bing, but their bot is no where to be found in activity stats. So it's pretty reasonable to think that at minimum their index is primarily derived from 3rd parties. https://deviceatlas.com/blog/most-active-bots-and-crawlers-web https://deviceatlas.com/blog/most-active-bots-and-crawlers-w... https://www.imperva.com/blog/most-active-good-bots/ https://www.imperva.com/blog/most-active-good-bots/ In my opinion, I don't think it should matter too much that they might not be crawling for their own index as long as they're reordering the results. But I do think there is some stigma where metasearch engines aren't looked at with as much prestige.
        the-rc 6y ago
        What kind of ranking can you do for a page on the internet if you didn't crawl its contents? It has to rely on other signals.
  - lights0123 6y ago
    I mean, searching something on DDG and then on Bing gives almost identical results.
    - Retric 6y ago
      Just did a search for tempest on bing, duck duck go, and google, and ddg was closer to google than bing. Which is fairly common, they are each the odd one out on some searches. Though I think google’s personalized search results make it seem very different to some people. Pick something really odd like “Purple car teacup” and google seems to give really bad results. 3 links to amazon, 2 to Pinterest, and 2 to EBay. They all link to thepurpleteacup.com, but only google has a Yoycart link, it also wants to replace car with cat. I think Bing started off with the best image search and ddg the worst, though bing wanted to ignore teacup for the bottom half of it’s results.
      - mda 6y ago
        Try a more complex query. e.g. "difference between wet and dry suit" bing and ddg web site results are almost identical, only ranking differences. Youtube results on the page are identical. Google is quite different. "average temperature of a dog" Again bing ddg very similar, Google completely different. "average lifespan of a domestic cat" "list of medication that has interactions with grapefruit". Just try more complex theories. The importance of ddg / bing relation is something they specifically and actively try to hide behind smoke and mirrors.
        Retric 6y ago
        “average temperature of a dog“ Bing and ddg are somewhat closer, but far from identical. Include the second page of googles results, and all three have vetinfo.com etc, but bing and google have two links to www.quora.com where ddg doesn’t have any. I only bring that up because of you compare results under the assumption that bing and ddg are identical it’s easy to overstate the similarities.
        philistine 6y ago
  - speedgoose 6y ago
    It's difficult to say as they are not open about it. Personally, I think they would suffer. Their API https://duckduckgo.com/api https://duckduckgo.com/api shows what they can provide as result. I believe the reason why it's missing a lot, most of the actual results, is because Microsoft Bing doesn't allow to offer a public API to redistribute the results from the Bing API.
- ffpip 6y ago
  That's my main concern. Duckduckgo is used more than Bing in many countries and Microsoft might just cancel their access some day. There is always Yandex I guess
  - luckylion 6y ago
    Why would Bing cancel, they're presumably making money off of the arrangement. And DDG-users would likely not migrate to Bing, but to Google: if you have to use a product by an anti-privacy-corporation, might as well use the good one.
    - ffpip 6y ago
      Well, the fact that DDG gets more users than Bing would be embarrassing for Bing.
      - willio58 6y ago
        Eh, was it embarrassing when Chrome was built on Webkit and they had more usage than Safari? I don't think so.
    - mistermann 6y ago
      Someone might strongly suggest the idea to them.
    - adammenges 6y ago
      Your google point is pretty valid, they’d be removing a source of income and sending users to a competitor. I think DDG is fine for the time being.
    - indigochill 6y ago
      >might as well use the good one While I do believe Google has the superior search engine, there may also be an argument for denying them the "vertical integration of identity". That is, if you use Gmail and Google Search, Google can figure out more about you (in theory) than if you use just Gmail. Same reason I never -ever- use Facebook SSO for logging into non-Facebook sites.
- chris_f 6y ago
  Gigablast, Mojeek, and Yandex (already a source of DDG results in Russia I believe) all have their own search indexes and APIs to access them. For a completely independent search engine, Gigabalst is surprisingly good in terms of index and relevance. Not as good as Google or Bing, but I think it is a one-man operation. Mojeek also just received an investment to build out their index. At Runnaroo, I partially use Google as the backend, but I have done a good amount of testing on the above options in case Google or Bing stop being options. Just as important as the search results, Bing provides the monetization for DDG through their ad network, so DDG is dependent on Bing for two critical parts of the business. I don't think Bing will ever kick them off because it must be very profitable for them and they need to do everything they can to get marketshare against Google.
- jahnu 6y ago
  https://www.qwant.com/ https://www.qwant.com/ Isn't bad
  - Dahoon 6y ago
    Better than DDG..
Angeo34 6y ago
So nobody will mention Mr. Weinberg business history. His last company used to accumulate private personal user info and then sell all to brokers and advertisers. DDG and privacy is a literal oxymoron just like Brave and privacy are. Google and Microsoft Edge are more private even though neither has any privacy.
- esperent 6y ago
  Maybe he's changed. Is there any evidence to suggest DDG is anything but honest?
  - Angeo34 6y ago
    I never got this logic. If it's about privacy they are obliged to proof that they are private especially with such a backstory. It's like Google saying "yes we are private" and then data share with the entirety of the US gov without any order. Why do you trust DDG did they do anything to earn your trust or is your trust that worthless?
    - esperent 6y ago
      I don't have any particular trust in them. However, they've staked their entire business model on privacy. That would be foolish if their intent is to sell user info, because doing that is unlikely to stay private for long.
      - ssorallen 6y ago
        Their entire business model is [selling advertisements][0], same as Google. [0]: https://help.duckduckgo.com/company/advertising-and-affiliates/ https://help.duckduckgo.com/company/advertising-and-affiliat...
        pbronez 6y ago
        DDG is using a very old version of Google’s business model: they serve ads based on your search string. That’s it. No search history mining. No email reading. No broadly deployed analytics snippets and browser logins to reconstruct your whole browsing history. I’m already sending my search query to DDG for them to give me search results. They don’t ask for any additional data. They just allow advertisers to bid on the first few results for a particular topic. I’m ok with that.
nwah1 6y ago
It is ironic that every new preference meant to enhance privacy only adds more bits of information by which to fingerprint you. But maybe fingerprinting is unavoidable. As far as I understand, to avoid fingerprinting, you would need to not have any browser cache at all.
- godelski 6y ago
  If DNT is a setting and respected by the website, how is this fingerprinting? My understanding is fingerprinting you need multiple metrics to compare to ensure someone is unique and thus you couldn't fingerprint purely based on a single setting (DNT) alone. (Obviously if it is not respected it becomes one more metric to add to enable better fingerprinting)
  - nitrogen 6y ago
    Check out EFF's Panopticlick demo. You basically answered your own question in your final parenthetical.
    - godelski 6y ago
      But part of this article is about giving DNT some teeth. The parenthetical statement was to show I understand that without the legal teeth it is another metric. But by ignoring the legal part you're ignoring the entire point of the post and my comment.
- capableweb 6y ago
  Yeah, strange that they don't focus on the legalities of the already existing dnt header, and instead added a new "Sec-GPC" header. https://globalprivacycontrol.github.io/gpc-spec/#the-sec-gpc-header-field-for-http-requests https://globalprivacycontrol.github.io/gpc-spec/#the-sec-gpc...
- contingencies 6y ago
  It's far from just the browser cache. Consider things like network location, download speed, interaction path, browser parsing edge cases, script enabled, privacy add-ons enabled, etc...
diqwndwqiond 6y ago
Is this a "commoditize your complements" thing?
beprogrammed 6y ago
I don't mean to discourage governments and groups trying to do what they can in regulating business. But surely privacy is a software feature not a corporate policy. Relying on others to not use the information at there disposal seems destined to fail.
deleted 6y ago
[deleted]
ozten 6y ago
“This would provide a key component that’s called for in the California law, which is a simple way for consumers to invoke their right without having to go to each website and find the button,” said Ashkan Soltani, a privacy researcher who helped lead the effort [ as well as the previous DNT effort ] "The new specification won’t become legally binding until the California attorney general blesses it." So it seems the started with the Law first, instead of Do not Track which tried to lead with code. https://www.wired.com/story/global-privacy-control-launches-do-not-track-is-back/ https://www.wired.com/story/global-privacy-control-launches-...
nuker 6y ago
> .. “do not sell or share” GPC signal What about storing, using for making user profiles, using for targeted ads ?

143 points by tagawa 6y ago

TekMol 6y ago

If Bing ever drops DuckDuckGo, will there be a search engine they can fall back on?

02020202 6y ago
they use bing only for some stuff, not as their backend. they will do just fine without bing.
- ffpip 6y ago
  Do you have any source? They have always avoided the question of how much they were dependent on Bing.
  - notRobot 6y ago
    AFAICT, other than those information boxes that turn up on the top of searches, the actual search results are almost entirely powerer by bing, although as you said, they've always dodged the question.
  - mrweasel 6y ago
    No one has a source for “DuckDuckGo is just Bing” either. We know that they use Bing, but not for what, people have just asserted that DDG is just Bing (or Yandex), but no one seems to be able to provide at source to back up the claim.
    - pb7 6y ago
      >To do that, DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (our crawler) and crowd-sourced sites (like Wikipedia, stored in our answer indexes). We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).[0] Seems pretty clear their standard search results come from Bing. It even used to say "multiple partners like Bing, Yahoo, etc" but Yahoo was also powered by Bing. They like to change the wording to make themselves sound less reliant on Bing even though if Bing were to cut off access, DuckDuckGo would immediately cease working. [0] https://help.duckduckgo.com/duckduckgo-help-pages/results/sources/ https://help.duckduckgo.com/duckduckgo-help-pages/results/so...
      - mrweasel 6y ago
        I’ve read that multiple times, and no, that’s not clear. At best we can read it as “Results from Bing are mixed into their search results”. Most likely a high percentage, but we don’t know.
        pb7 6y ago
        Mission accomplished then because they don't want to make it clear. If you really care that much, go back through all the different revisions of that statement through the Internet Archive and see how many times they've been dancing around saying the exact same thing: it's just fucking Bing. Their own crawler returns results for very specific limited topics in their segregated boxes. If they had even one source for one specific query, say "purple cats" that they didn't use Bing for, they could technically claim it's only "mostly" Bing.
    - sct202 6y ago
      We can infer some things about DDG based published data on crawler and bot activity. DDG's index is pretty comparable to both Google and Bing, but their bot is no where to be found in activity stats. So it's pretty reasonable to think that at minimum their index is primarily derived from 3rd parties. https://deviceatlas.com/blog/most-active-bots-and-crawlers-web https://deviceatlas.com/blog/most-active-bots-and-crawlers-w... https://www.imperva.com/blog/most-active-good-bots/ https://www.imperva.com/blog/most-active-good-bots/ In my opinion, I don't think it should matter too much that they might not be crawling for their own index as long as they're reordering the results. But I do think there is some stigma where metasearch engines aren't looked at with as much prestige.
      - the-rc 6y ago
        What kind of ranking can you do for a page on the internet if you didn't crawl its contents? It has to rely on other signals.
- lights0123 6y ago
  I mean, searching something on DDG and then on Bing gives almost identical results.
  - Retric 6y ago
    Just did a search for tempest on bing, duck duck go, and google, and ddg was closer to google than bing. Which is fairly common, they are each the odd one out on some searches. Though I think google’s personalized search results make it seem very different to some people. Pick something really odd like “Purple car teacup” and google seems to give really bad results. 3 links to amazon, 2 to Pinterest, and 2 to EBay. They all link to thepurpleteacup.com, but only google has a Yoycart link, it also wants to replace car with cat. I think Bing started off with the best image search and ddg the worst, though bing wanted to ignore teacup for the bottom half of it’s results.
    - mda 6y ago
      Try a more complex query. e.g. "difference between wet and dry suit" bing and ddg web site results are almost identical, only ranking differences. Youtube results on the page are identical. Google is quite different. "average temperature of a dog" Again bing ddg very similar, Google completely different. "average lifespan of a domestic cat" "list of medication that has interactions with grapefruit". Just try more complex theories. The importance of ddg / bing relation is something they specifically and actively try to hide behind smoke and mirrors.
      - Retric 6y ago
        “average temperature of a dog“ Bing and ddg are somewhat closer, but far from identical. Include the second page of googles results, and all three have vetinfo.com etc, but bing and google have two links to www.quora.com where ddg doesn’t have any. I only bring that up because of you compare results under the assumption that bing and ddg are identical it’s easy to overstate the similarities.
        philistine 6y ago
- speedgoose 6y ago
  It's difficult to say as they are not open about it. Personally, I think they would suffer. Their API https://duckduckgo.com/api https://duckduckgo.com/api shows what they can provide as result. I believe the reason why it's missing a lot, most of the actual results, is because Microsoft Bing doesn't allow to offer a public API to redistribute the results from the Bing API.
ffpip 6y ago
That's my main concern. Duckduckgo is used more than Bing in many countries and Microsoft might just cancel their access some day. There is always Yandex I guess
- luckylion 6y ago
  Why would Bing cancel, they're presumably making money off of the arrangement. And DDG-users would likely not migrate to Bing, but to Google: if you have to use a product by an anti-privacy-corporation, might as well use the good one.
  - ffpip 6y ago
    Well, the fact that DDG gets more users than Bing would be embarrassing for Bing.
    - willio58 6y ago
      Eh, was it embarrassing when Chrome was built on Webkit and they had more usage than Safari? I don't think so.
  - mistermann 6y ago
    Someone might strongly suggest the idea to them.
  - adammenges 6y ago
    Your google point is pretty valid, they’d be removing a source of income and sending users to a competitor. I think DDG is fine for the time being.
  - indigochill 6y ago
    >might as well use the good one While I do believe Google has the superior search engine, there may also be an argument for denying them the "vertical integration of identity". That is, if you use Gmail and Google Search, Google can figure out more about you (in theory) than if you use just Gmail. Same reason I never -ever- use Facebook SSO for logging into non-Facebook sites.
chris_f 6y ago
Gigablast, Mojeek, and Yandex (already a source of DDG results in Russia I believe) all have their own search indexes and APIs to access them. For a completely independent search engine, Gigabalst is surprisingly good in terms of index and relevance. Not as good as Google or Bing, but I think it is a one-man operation. Mojeek also just received an investment to build out their index. At Runnaroo, I partially use Google as the backend, but I have done a good amount of testing on the above options in case Google or Bing stop being options. Just as important as the search results, Bing provides the monetization for DDG through their ad network, so DDG is dependent on Bing for two critical parts of the business. I don't think Bing will ever kick them off because it must be very profitable for them and they need to do everything they can to get marketshare against Google.
jahnu 6y ago
https://www.qwant.com/ https://www.qwant.com/ Isn't bad
- Dahoon 6y ago
  Better than DDG..

Angeo34 6y ago

So nobody will mention Mr. Weinberg business history. His last company used to accumulate private personal user info and then sell all to brokers and advertisers. DDG and privacy is a literal oxymoron just like Brave and privacy are. Google and Microsoft Edge are more private even though neither has any privacy.

esperent 6y ago
Maybe he's changed. Is there any evidence to suggest DDG is anything but honest?
- Angeo34 6y ago
  I never got this logic. If it's about privacy they are obliged to proof that they are private especially with such a backstory. It's like Google saying "yes we are private" and then data share with the entirety of the US gov without any order. Why do you trust DDG did they do anything to earn your trust or is your trust that worthless?
  - esperent 6y ago
    I don't have any particular trust in them. However, they've staked their entire business model on privacy. That would be foolish if their intent is to sell user info, because doing that is unlikely to stay private for long.
    - ssorallen 6y ago
      Their entire business model is [selling advertisements][0], same as Google. [0]: https://help.duckduckgo.com/company/advertising-and-affiliates/ https://help.duckduckgo.com/company/advertising-and-affiliat...
      - pbronez 6y ago
        DDG is using a very old version of Google’s business model: they serve ads based on your search string. That’s it. No search history mining. No email reading. No broadly deployed analytics snippets and browser logins to reconstruct your whole browsing history. I’m already sending my search query to DDG for them to give me search results. They don’t ask for any additional data. They just allow advertisers to bid on the first few results for a particular topic. I’m ok with that.

nwah1 6y ago

It is ironic that every new preference meant to enhance privacy only adds more bits of information by which to fingerprint you. But maybe fingerprinting is unavoidable. As far as I understand, to avoid fingerprinting, you would need to not have any browser cache at all.

godelski 6y ago
If DNT is a setting and respected by the website, how is this fingerprinting? My understanding is fingerprinting you need multiple metrics to compare to ensure someone is unique and thus you couldn't fingerprint purely based on a single setting (DNT) alone. (Obviously if it is not respected it becomes one more metric to add to enable better fingerprinting)
- nitrogen 6y ago
  Check out EFF's Panopticlick demo. You basically answered your own question in your final parenthetical.
  - godelski 6y ago
    But part of this article is about giving DNT some teeth. The parenthetical statement was to show I understand that without the legal teeth it is another metric. But by ignoring the legal part you're ignoring the entire point of the post and my comment.
capableweb 6y ago
Yeah, strange that they don't focus on the legalities of the already existing dnt header, and instead added a new "Sec-GPC" header. https://globalprivacycontrol.github.io/gpc-spec/#the-sec-gpc-header-field-for-http-requests https://globalprivacycontrol.github.io/gpc-spec/#the-sec-gpc...
contingencies 6y ago
It's far from just the browser cache. Consider things like network location, download speed, interaction path, browser parsing edge cases, script enabled, privacy add-ons enabled, etc...

diqwndwqiond 6y ago

Is this a "commoditize your complements" thing?

beprogrammed 6y ago

I don't mean to discourage governments and groups trying to do what they can in regulating business. But surely privacy is a software feature not a corporate policy. Relying on others to not use the information at there disposal seems destined to fail.

deleted 6y ago

[deleted]

ozten 6y ago

“This would provide a key component that’s called for in the California law, which is a simple way for consumers to invoke their right without having to go to each website and find the button,” said Ashkan Soltani, a privacy researcher who helped lead the effort [ as well as the previous DNT effort ] "The new specification won’t become legally binding until the California attorney general blesses it." So it seems the started with the Law first, instead of Do not Track which tried to lead with code. https://www.wired.com/story/global-privacy-control-launches-do-not-track-is-back/ https://www.wired.com/story/global-privacy-control-launches-...

nuker 6y ago

> .. “do not sell or share” GPC signal What about storing, using for making user profiles, using for targeted ads ?