9 ms·

Measuring open DNS resolver use

47 points by xuande 7y ago

gerdesj 7y ago
The argument is well known: the intertubes promises much equanimity but capitalism and stuff. OK I am being a bit cruel but this is what we have. If you don't own up to intending to cuddle up to Amazon, Google, Apple, Microsoft in the next 30s then you are probably a liar or a bit deluded. Thing is, I'm a bit of a fan of capitalism but perhaps some sort of light touch regulation is needed in the Wild West. A bloke with a big old star on the chest might be nice.
- pnako 7y ago
  It's probably even worse than what you describe. You'll be cuddling up to the Amazon, Google, Apple, Microsoft of whichever global power you're closest to. I'm in Australia, so I don't know if in twenty years I'll still be connected to Westnet or Sinonet. I'll probably buy a black market connection to Westnet from a guy with a mohawk and implants in his head.
  - troquerre 7y ago
    Agreed. The world is trending towards an internet that's split up between the different countries. There are some technologies that can counter that (ie check my bio) but it'll be an uphill battle imo.
PaulHoule 7y ago
I think DNS-over-HTTPS is a much bigger cause for concern.
- cracker_jacks 7y ago
  What is the cause for concern? I am trying to understand why DNS-over-HTTPS could be a bad thing from the user end.
  - tptacek 7y ago
    It's not. It's an unalloyed good thing. The concern is with configurations that make it hard to use anything but Cloud Flare. There are alternatives.
    - regecks 7y ago
      What do you think about Paul Vixie's views on DoH? On the face of it, it seems like we're going to end up with a bunch of black box devices (from Google, Amazon etc) in our homes that are totally immune to most forms of network policing because between DoH, ESNI, TLS and CDN fronting, you can't see anything.
      - tptacek 7y ago
        Paul Vixie has been one of the Internet's most dedicated proponents of DNSSEC, a technology that essentially escrows keys with governments. If DNSSEC and DANE had progressed according to Vixie's preferred schedule, Muammar Gaddafi would have owned BIT.LY's CA. Vixie operates a company that relies on passive DNS observation to generate telemetry for corporations. Smart dude. Would not weight his privacy opinions heavily.
        tialaramex 7y ago
        DNSSEC doesn't "essentially escrow keys with governments". It's exactly as true to say that DANE gave Gaddafi ownership of bit.ly's CA as to say that today Boris Johnson owns the CA for slither.io - and as ridiculous. Back when you first started claiming this the Ten Blessed Methods weren't even a thing. You're complaining about the inadequate back door lock on a house that has the front door propped open. I work for an outfit that buys passive DNS data (among other things) and all our suppliers agree that DoH makes no difference to their roadmaps - because they don't care who asked, only what the query and answer are. Is Vixie doing something vastly different? Maybe so but you've offered no evidence what that could be.
        tptacek 7y ago
mike_d 7y ago
Folks here might be interested in my own (more tl;dr) survey of public resolver usage: https://twitter.com/mikedamm/status/1136409254305263616?s=20 https://twitter.com/mikedamm/status/1136409254305263616?s=20
- troquerre 7y ago
  Do you know how Cloudflare picked up so much market share even with 8.8.8.8 around?
  - ssvss 7y ago
    Google doesn’t promote 8.8.x.x lately, so when there were some sites being blocked by Indian ISPs, I saw cloudflare DNS being recommended. Also I believe cloudflare doesn’t implement ban-lists provided by Indian courts. I have noticed sites blocked in google dns, working with cloudflare dns.
    - jackcodes 7y ago
      To extend this, I’ve never seen google promote 8.8 publicly. All recommendations I’ve seen have been from other users or forum members. It maybe that Google’s trust isn’t what it once was, causing a reduction in recommendation.
kijin 7y ago
What this data shows is that concerns about centralization, especially in relation to DoH, are overblown. Only 1.15% of users in this dataset are using Cloudflare DNS, and APNIC is in a region riddled with government censorship and crappy ISPs -- two major incentives for people to try alternative resolvers. Without such incentives, nobody would even bother to change their devices' DNS settings. I'm in a country with both of these problems, so every machine I set up gets Cloudflare as the primary resolver with Google as the secondary. Fix these problems first, and I won't have to do this anymore. Centralization? I dunno. Taking control of DNS away from my state and ISP would actually count as decentralization in my book.
- troquerre 7y ago
  Agree that shifting control from your state and ISP are beneficial, but centralizing DNS to one or two for-profit providers in the process is less than ideal IMO. I'm working on a project that's aiming to make DNS fully decentralized and wrote an explainer article in case you're interested https://www.namebase.io/blog/meet-handshake-decentralizing-dns-to-improve-the-security-of-the-internet/ https://www.namebase.io/blog/meet-handshake-decentralizing-d... If it's not too revealing, can you share what country you're in?
  - kijin 7y ago
    Sure, even more decentralization would be good. Nevertheless, I think the usual concerns about Cloudflare are massively overblown and misses a crucial role that they're playing in the fight against censorship and surveillance. Cloudflare is the first well-known provider that decided to support a working protocol for encrypted DNS. DoH might not be the best possible protocol, but it's shipping now and others are not. (DNSCrypt is also shipping, but it has the weird property of speaking something that isn't HTTPS on port 443. That's too easy to censor.) There will be healthy competition if other people would please stop arguing and start shipping, too. I'm in South Korea. The censorship regime here is more prudish than draconian, and I'm not in any danger of prosecution for criticizing it. The way it is implemented is extremely crappy, though. On top of DNS-based censorship, ISPs are doing DPI on TLS handshakes to sniff hostnames in the SNI extension. Lots of techies here are very interested in new technologies showcased by Cloudflare. That includes not only DoH but also their proposal to use keys stored in DNS to close the SNI loophole. Once again, what works in this situation is not a novel, theoretically perfect protocol but one that ships now and blends into the petabytes of HTTPS traffic that Cloudflare handles every day.
amalcon 7y ago
I've worked on software that attempts to estimate the end-user demand represented by a given resolver. Most of the specifics are covered by NDA, but suffice to say that it's surprisingly difficult to do accurately. There is a lot of non-standard behavior in various resolver software, some of it avoidable, some of it not. This trick of altering the hostname of a subresource to identify the client is a thing I long wished we could do, but sadly we didn't have enough control of the content to do that.
3xblah 7y ago
How about measuring zone file access programs? How many users utilise the full number of existing domain names in the world today? How many names do users realistically need to access in a lifetime? What if we exclude ad servers and other domains that exist solely for marketing? It depends on the user, but in some cases the majority of their non-commercial web^1 use can be accomplished without ever making remote DNS queries; the IP addresses can be stored and used on a long-term basis. That is because a user may only visit the same small number of websites. The foregoing is of course only an opinion based on testing conducted by yours truly. Every user is different. Try measuring how many times the non-commercial websites you visit change IP addresses in a year. You might find that DNS resolution is like that "definition of insanity" meme: making the same query day after day, expecting a different answer. 1 It makes a difference whether or not a user is using the web to make purchases. For making purchases online, DNS resolution is almost always required. Domains and IP addresses in that context are constantly changing. Go figure. OTOH, if you are visiting a website such as news.ycombinator.com on a frequent basis, is it really necessary to look up the IP address for news.ycombinator.com every time you visit the website? I have used the same IP address for years at a time.

47 points by xuande 7y ago

The argument is well known: the intertubes promises much equanimity but capitalism and stuff. OK I am being a bit cruel but this is what we have. If you don't own up to intending to cuddle up to Amazon, Google, Apple, Microsoft in the next 30s then you are probably a liar or a bit deluded. Thing is, I'm a bit of a fan of capitalism but perhaps some sort of light touch regulation is needed in the Wild West. A bloke with a big old star on the chest might be nice.

pnako 7y ago
It's probably even worse than what you describe. You'll be cuddling up to the Amazon, Google, Apple, Microsoft of whichever global power you're closest to. I'm in Australia, so I don't know if in twenty years I'll still be connected to Westnet or Sinonet. I'll probably buy a black market connection to Westnet from a guy with a mohawk and implants in his head.
- troquerre 7y ago
  Agreed. The world is trending towards an internet that's split up between the different countries. There are some technologies that can counter that (ie check my bio) but it'll be an uphill battle imo.

PaulHoule 7y ago

I think DNS-over-HTTPS is a much bigger cause for concern.

cracker_jacks 7y ago
What is the cause for concern? I am trying to understand why DNS-over-HTTPS could be a bad thing from the user end.
- tptacek 7y ago
  It's not. It's an unalloyed good thing. The concern is with configurations that make it hard to use anything but Cloud Flare. There are alternatives.
  - regecks 7y ago
    What do you think about Paul Vixie's views on DoH? On the face of it, it seems like we're going to end up with a bunch of black box devices (from Google, Amazon etc) in our homes that are totally immune to most forms of network policing because between DoH, ESNI, TLS and CDN fronting, you can't see anything.
    - tptacek 7y ago
      Paul Vixie has been one of the Internet's most dedicated proponents of DNSSEC, a technology that essentially escrows keys with governments. If DNSSEC and DANE had progressed according to Vixie's preferred schedule, Muammar Gaddafi would have owned BIT.LY's CA. Vixie operates a company that relies on passive DNS observation to generate telemetry for corporations. Smart dude. Would not weight his privacy opinions heavily.
      - tialaramex 7y ago
        DNSSEC doesn't "essentially escrow keys with governments". It's exactly as true to say that DANE gave Gaddafi ownership of bit.ly's CA as to say that today Boris Johnson owns the CA for slither.io - and as ridiculous. Back when you first started claiming this the Ten Blessed Methods weren't even a thing. You're complaining about the inadequate back door lock on a house that has the front door propped open. I work for an outfit that buys passive DNS data (among other things) and all our suppliers agree that DoH makes no difference to their roadmaps - because they don't care who asked, only what the query and answer are. Is Vixie doing something vastly different? Maybe so but you've offered no evidence what that could be.
        tptacek 7y ago

mike_d 7y ago

Folks here might be interested in my own (more tl;dr) survey of public resolver usage: https://twitter.com/mikedamm/status/1136409254305263616?s=20 https://twitter.com/mikedamm/status/1136409254305263616?s=20

troquerre 7y ago
Do you know how Cloudflare picked up so much market share even with 8.8.8.8 around?
- ssvss 7y ago
  Google doesn’t promote 8.8.x.x lately, so when there were some sites being blocked by Indian ISPs, I saw cloudflare DNS being recommended. Also I believe cloudflare doesn’t implement ban-lists provided by Indian courts. I have noticed sites blocked in google dns, working with cloudflare dns.
  - jackcodes 7y ago
    To extend this, I’ve never seen google promote 8.8 publicly. All recommendations I’ve seen have been from other users or forum members. It maybe that Google’s trust isn’t what it once was, causing a reduction in recommendation.

kijin 7y ago

What this data shows is that concerns about centralization, especially in relation to DoH, are overblown. Only 1.15% of users in this dataset are using Cloudflare DNS, and APNIC is in a region riddled with government censorship and crappy ISPs -- two major incentives for people to try alternative resolvers. Without such incentives, nobody would even bother to change their devices' DNS settings. I'm in a country with both of these problems, so every machine I set up gets Cloudflare as the primary resolver with Google as the secondary. Fix these problems first, and I won't have to do this anymore. Centralization? I dunno. Taking control of DNS away from my state and ISP would actually count as decentralization in my book.

troquerre 7y ago
Agree that shifting control from your state and ISP are beneficial, but centralizing DNS to one or two for-profit providers in the process is less than ideal IMO. I'm working on a project that's aiming to make DNS fully decentralized and wrote an explainer article in case you're interested https://www.namebase.io/blog/meet-handshake-decentralizing-dns-to-improve-the-security-of-the-internet/ https://www.namebase.io/blog/meet-handshake-decentralizing-d... If it's not too revealing, can you share what country you're in?
- kijin 7y ago
  Sure, even more decentralization would be good. Nevertheless, I think the usual concerns about Cloudflare are massively overblown and misses a crucial role that they're playing in the fight against censorship and surveillance. Cloudflare is the first well-known provider that decided to support a working protocol for encrypted DNS. DoH might not be the best possible protocol, but it's shipping now and others are not. (DNSCrypt is also shipping, but it has the weird property of speaking something that isn't HTTPS on port 443. That's too easy to censor.) There will be healthy competition if other people would please stop arguing and start shipping, too. I'm in South Korea. The censorship regime here is more prudish than draconian, and I'm not in any danger of prosecution for criticizing it. The way it is implemented is extremely crappy, though. On top of DNS-based censorship, ISPs are doing DPI on TLS handshakes to sniff hostnames in the SNI extension. Lots of techies here are very interested in new technologies showcased by Cloudflare. That includes not only DoH but also their proposal to use keys stored in DNS to close the SNI loophole. Once again, what works in this situation is not a novel, theoretically perfect protocol but one that ships now and blends into the petabytes of HTTPS traffic that Cloudflare handles every day.

amalcon 7y ago

I've worked on software that attempts to estimate the end-user demand represented by a given resolver. Most of the specifics are covered by NDA, but suffice to say that it's surprisingly difficult to do accurately. There is a lot of non-standard behavior in various resolver software, some of it avoidable, some of it not. This trick of altering the hostname of a subresource to identify the client is a thing I long wished we could do, but sadly we didn't have enough control of the content to do that.

3xblah 7y ago

How about measuring zone file access programs? How many users utilise the full number of existing domain names in the world today? How many names do users realistically need to access in a lifetime? What if we exclude ad servers and other domains that exist solely for marketing? It depends on the user, but in some cases the majority of their non-commercial web^1 use can be accomplished without ever making remote DNS queries; the IP addresses can be stored and used on a long-term basis. That is because a user may only visit the same small number of websites. The foregoing is of course only an opinion based on testing conducted by yours truly. Every user is different. Try measuring how many times the non-commercial websites you visit change IP addresses in a year. You might find that DNS resolution is like that "definition of insanity" meme: making the same query day after day, expecting a different answer. 1 It makes a difference whether or not a user is using the web to make purchases. For making purchases online, DNS resolution is almost always required. Domains and IP addresses in that context are constantly changing. Go figure. OTOH, if you are visiting a website such as news.ycombinator.com on a frequent basis, is it really necessary to look up the IP address for news.ycombinator.com every time you visit the website? I have used the same IP address for years at a time.