10 ms·

I think a better link to explain is here: https://duck.co/help/results/yahoo-technical-implementation https://duck.co/help/results/yahoo-technical-implementatio

by pwnna 10y ago

I think a better link to explain is here: https://duck.co/help/results/yahoo-technical-implementation https://duck.co/help/results/yahoo-technical-implementation It looks like they are just querying yahoo for results and ads as a proxy and it sounds like, through some contractual/technical limitations, they have to make that request from a yahoo controlled domain name (duckduckgo-owned-server.yahoo.net). However, they say they fully control the server and yahoo does not get to touch it, as yahoo delegated DNS to them. The change does not seem malicious and if what they say is true, then the implementation might not be too bad. The volume of traffic they get probably means you cannot profile an individual from Yahoo's side. However, this potentially means that your search history is leaked to Yahoo, albeit in an aggregate manner with other DDG users, which may have attacks that I'm not aware of. What I can see, however, is a lot of noscript users get startled by this change as they will see DDG use a script from yahoo.net (actually duckduckgo-owned-server.yahoo.net) as by default noscript does not show the full domain.

robszumski 10y ago
So they data mine this traffic at the nameserver level, just to reasonably say that DDG doesn't share info? Making it technically your computer sharing the info, by way of their forced implementation? Seems like the ad blockers are making the right move here.
- jsmthrowaway 10y ago
  This comment is puzzling, since authoritative nameservers only see actual end-user traffic on an occasional basis given the cornucopia of resolver caches in near-unanimous use (as well as the fat TTL on the yahoo.net side of the delegation). This denies Yahoo! the reliable data that you seem to think they've successfully pulled over on everyone, at least if I understand your comment correctly which I'll admit I'm having a hard time doing. It's also an odd concern given that most everybody these days uses Google or their ISP's resolvers which would be far more interesting "nameserver level data mining." (I have a hard time believing Google successfully logs that traffic in detail, however.)
  - madeofpalk 10y ago
    Just being devils advocate here, but wouldn't the cache rate be more or less uniform and able to be interpolated out into real traffic numbers? If yahoo observes that every 1 in 3 yahoo.com requests in NS lookups then they could still figure out the traffic? In practice.... i doubt this is valuable data at all.
    - jsmthrowaway 10y ago
      I can't grok what you're asking me but I'm almost certain the answer is no, as you've already inferred. Extrapolating DNS analytics to actionable intel is incredibly misleading in nearly every case on the service side, particularly since there are a number of caches that consider TTL a guideline and since large shared caches are commonplace. Understanding what your company is doing if you run the resolver is where it makes more sense. But on the service side? You might as well hire a cat to report metrics by pawing a ouija board, and then you at least get cuddles with your random numbers. Think of it this way: DNS is basically a different form of ARP, in a certain manner of speaking. I wouldn't try to quantify performance data from either except to troubleshoot operations.
- nl 10y ago
  So they data mine this traffic at the nameserver level, just to reasonably say that DDG doesn't share info? What does this mean? If you mean nameservers might get to see your IP, that isn't true. At most they'll see your primary DNS resolver's IP, but if you use Google DNS or Open DNS they will only see that (I've implemented CDNs, and this is a very big problem for them if they operate at the DNS level - which most do).
  - ForHackernews 10y ago
    > if you use Google DNS > Afraid to give Google my search queries >> I'll give them all of my DNS lookups instead
    - nl 10y ago
      So... don't? Just use your default ISP resolvers? Depending on what threat you are trying to avoid there are options here. The OPs threat assessment was wrong, though.
piyush_soni 10y ago
The cynic in me says that it all starts like this. With multiple, gradual ToS changes, eventually, it will become like any other company out there. And what if Yahoo acquires DDG?
- angersock 10y ago
  Then one of us can try to break into the search market! :D
  - joe563323 10y ago
    privacy first search market
- beardog 10y ago
  That would be very foolish of duckduckgo, their whole niche is privacy.
- volatilitish 10y ago
  Isn't that basically the MO of every startup? * Offer something for free, with VC money. * Gain market share since you don't need to worry about profit. * Realise you need to make money, so start turning into the companies you stole market share from.
  - tagawa 10y ago
    Just want to point out that although that may be a model for many startups, DuckDuckGo is already profitable through (non-tracking) advertising and affiliate revenue. More info: https://duck.co/help/company/advertising-and-affiliates https://duck.co/help/company/advertising-and-affiliates Disclaimer: DDG staff
    - youdontknowtho 10y ago
      That's great news, and probably needs to be more widely known.
    - 10y ago
jsingleton 10y ago
That link is pretty good at explaining the DNS delegation, TLS and NoScript. However, as DDG runs on AWS, they don't have sole control of their servers anyway. Edit: output of a DNS/Ping tool I wrote for a book: Enter a hostname or IP address: duckduckgo.com Performing DNS lookup of duckduckgo.com Complete, duckduckgo.com = 54.229.105.92 Performing reverse DNS lookup of 54.229.105.92 Complete, 54.229.105.92 = ec2-54-229-105-92.eu-west-1.compute.amazonaws.com Complete, duckduckgo.com = 54.229.105.203 Performing reverse DNS lookup of 54.229.105.203 Complete, 54.229.105.203 = ec2-54-229-105-203.eu-west-1.compute.amazonaws.com Complete, duckduckgo.com = 46.51.197.89 Performing reverse DNS lookup of 46.51.197.89 Complete, 46.51.197.89 = ec2-46-51-197-89.eu-west-1.compute.amazonaws.com Complete, duckduckgo.com = 176.34.135.167 Performing reverse DNS lookup of 176.34.135.167 Complete, 176.34.135.167 = ec2-176-34-135-167.eu-west-1.compute.amazonaws.com Complete, duckduckgo.com = 176.34.155.20 Performing reverse DNS lookup of 176.34.155.20 Complete, 176.34.155.20 = ec2-176-34-155-20.eu-west-1.compute.amazonaws.com Complete, duckduckgo.com = 176.34.131.233 Performing reverse DNS lookup of 176.34.131.233 Complete, 176.34.131.233 = ec2-176-34-131-233.eu-west-1.compute.amazonaws.com Pinging duckduckgo.com 4 times Ping attempt #1 of 4 Success 30 ms Ping attempt #2 of 4 Success 30 ms Ping attempt #3 of 4 Success 30 ms Ping attempt #4 of 4 Success 29 ms Press any key to exit... Source code (C#): https://github.com/PacktPublishing/ASP.NET-Core-1.0-High-Performance/tree/master/Chapter%205/DnsPingTool https://github.com/PacktPublishing/ASP.NET-Core-1.0-High-Per...
- djsumdog 10y ago
  TIL! I didn't realize they ran an AWS stack. I've been using DDG for years (use !g about 1/3 of the time I think), and I just like having the alternative. I wish we had more search engines, but I also realize the barrier to entry is really high. Using AWS does make a lot of sense in their market. I thought their primary index was Yandex though, and they purchased supplemental data from Yahoo and Google?
  - pttrsmrt 10y ago
    Supertip: Use !sp instead of !g to get googles results but without the tracking.

by pwnna 10y ago

robszumski 10y ago

So they data mine this traffic at the nameserver level, just to reasonably say that DDG doesn't share info? Making it technically your computer sharing the info, by way of their forced implementation? Seems like the ad blockers are making the right move here.

jsmthrowaway 10y ago
This comment is puzzling, since authoritative nameservers only see actual end-user traffic on an occasional basis given the cornucopia of resolver caches in near-unanimous use (as well as the fat TTL on the yahoo.net side of the delegation). This denies Yahoo! the reliable data that you seem to think they've successfully pulled over on everyone, at least if I understand your comment correctly which I'll admit I'm having a hard time doing. It's also an odd concern given that most everybody these days uses Google or their ISP's resolvers which would be far more interesting "nameserver level data mining." (I have a hard time believing Google successfully logs that traffic in detail, however.)
- madeofpalk 10y ago
  Just being devils advocate here, but wouldn't the cache rate be more or less uniform and able to be interpolated out into real traffic numbers? If yahoo observes that every 1 in 3 yahoo.com requests in NS lookups then they could still figure out the traffic? In practice.... i doubt this is valuable data at all.
  - jsmthrowaway 10y ago
    I can't grok what you're asking me but I'm almost certain the answer is no, as you've already inferred. Extrapolating DNS analytics to actionable intel is incredibly misleading in nearly every case on the service side, particularly since there are a number of caches that consider TTL a guideline and since large shared caches are commonplace. Understanding what your company is doing if you run the resolver is where it makes more sense. But on the service side? You might as well hire a cat to report metrics by pawing a ouija board, and then you at least get cuddles with your random numbers. Think of it this way: DNS is basically a different form of ARP, in a certain manner of speaking. I wouldn't try to quantify performance data from either except to troubleshoot operations.
nl 10y ago
So they data mine this traffic at the nameserver level, just to reasonably say that DDG doesn't share info? What does this mean? If you mean nameservers might get to see your IP, that isn't true. At most they'll see your primary DNS resolver's IP, but if you use Google DNS or Open DNS they will only see that (I've implemented CDNs, and this is a very big problem for them if they operate at the DNS level - which most do).
- ForHackernews 10y ago
  > if you use Google DNS > Afraid to give Google my search queries >> I'll give them all of my DNS lookups instead
  - nl 10y ago
    So... don't? Just use your default ISP resolvers? Depending on what threat you are trying to avoid there are options here. The OPs threat assessment was wrong, though.

piyush_soni 10y ago

The cynic in me says that it all starts like this. With multiple, gradual ToS changes, eventually, it will become like any other company out there. And what if Yahoo acquires DDG?

angersock 10y ago
Then one of us can try to break into the search market! :D
- joe563323 10y ago
  privacy first search market
beardog 10y ago
That would be very foolish of duckduckgo, their whole niche is privacy.
volatilitish 10y ago
Isn't that basically the MO of every startup? * Offer something for free, with VC money. * Gain market share since you don't need to worry about profit. * Realise you need to make money, so start turning into the companies you stole market share from.
- tagawa 10y ago
  Just want to point out that although that may be a model for many startups, DuckDuckGo is already profitable through (non-tracking) advertising and affiliate revenue. More info: https://duck.co/help/company/advertising-and-affiliates https://duck.co/help/company/advertising-and-affiliates Disclaimer: DDG staff
  - youdontknowtho 10y ago
    That's great news, and probably needs to be more widely known.
  - 10y ago

jsingleton 10y ago

That link is pretty good at explaining the DNS delegation, TLS and NoScript. However, as DDG runs on AWS, they don't have sole control of their servers anyway. Edit: output of a DNS/Ping tool I wrote for a book: Enter a hostname or IP address: duckduckgo.com Performing DNS lookup of duckduckgo.com Complete, duckduckgo.com = 54.229.105.92 Performing reverse DNS lookup of 54.229.105.92 Complete, 54.229.105.92 = ec2-54-229-105-92.eu-west-1.compute.amazonaws.com Complete, duckduckgo.com = 54.229.105.203 Performing reverse DNS lookup of 54.229.105.203 Complete, 54.229.105.203 = ec2-54-229-105-203.eu-west-1.compute.amazonaws.com Complete, duckduckgo.com = 46.51.197.89 Performing reverse DNS lookup of 46.51.197.89 Complete, 46.51.197.89 = ec2-46-51-197-89.eu-west-1.compute.amazonaws.com Complete, duckduckgo.com = 176.34.135.167 Performing reverse DNS lookup of 176.34.135.167 Complete, 176.34.135.167 = ec2-176-34-135-167.eu-west-1.compute.amazonaws.com Complete, duckduckgo.com = 176.34.155.20 Performing reverse DNS lookup of 176.34.155.20 Complete, 176.34.155.20 = ec2-176-34-155-20.eu-west-1.compute.amazonaws.com Complete, duckduckgo.com = 176.34.131.233 Performing reverse DNS lookup of 176.34.131.233 Complete, 176.34.131.233 = ec2-176-34-131-233.eu-west-1.compute.amazonaws.com Pinging duckduckgo.com 4 times Ping attempt #1 of 4 Success 30 ms Ping attempt #2 of 4 Success 30 ms Ping attempt #3 of 4 Success 30 ms Ping attempt #4 of 4 Success 29 ms Press any key to exit... Source code (C#): https://github.com/PacktPublishing/ASP.NET-Core-1.0-High-Performance/tree/master/Chapter%205/DnsPingTool https://github.com/PacktPublishing/ASP.NET-Core-1.0-High-Per...

djsumdog 10y ago
TIL! I didn't realize they ran an AWS stack. I've been using DDG for years (use !g about 1/3 of the time I think), and I just like having the alternative. I wish we had more search engines, but I also realize the barrier to entry is really high. Using AWS does make a lot of sense in their market. I thought their primary index was Yandex though, and they purchased supplemental data from Yahoo and Google?
- pttrsmrt 10y ago
  Supertip: Use !sp instead of !g to get googles results but without the tracking.