4 ms·

What an arrogant hack. It's hard to believe Google engineers would find that a reasonable approach. If I'd have seen that during a code review, I would have cal

What an arrogant hack. It's hard to believe Google engineers would find that a reasonable approach. If I'd have seen that during a code review, I would have called it out and explained that you shouldn't abuse someone else's systems. Then again, maybe it's revenge for everyone pinging 8.8.8.8

deleted 4y ago
[deleted]
- deleted 4y ago
  [deleted]
mschuster91 4y ago
The problem is, there is no better approach to check for intercepting middleboxes. Using a well-known path (like gstatic.com/generate_204) works for detecting if the user has a captive portal between them and the Internet, but not if the user's provider messes around with DNS.
- blueflow 4y ago
  Google could host their own DNS servers who only give out NXD and test against that.
  - mschuster91 4y ago
    As soon as Google would do that, providers would implement exceptions.
    - adql 4y ago
      They could do DPI, allow traffic for unknown domains, intercept everything else. Differentiating random domain from "what user might actually type by accident" is also pretty trivial
- inetknght 4y ago
  > there is no better approach to check for intercepting middleboxes I can see some purposes for detecting middleboxes. I've done it. It usually doesn't involve DNS. It does involve certificate pinning though. > detecting if the user has a captive portal between them and the Internet That's easy. Try to browse to something. If succeeds but the certificate isn't valid then the user probably has a captive portal. That, or your pinned certificate has been revoked. > detecting ... if the user's provider messes around with DNS Certificate pinning, again, comes to the rescue. Pin a certificate to your own DoH server and then use DoH to look up whatever you need. If you can't connect to your DoH server then you effectively aren't (or shouldn't be) connected to the internet.
  - opheliate 4y ago
    The kind of interception that is being discussed here is the interception of DNS queries which would return an NXDOMAIN result to instead point to an advertising page. This is done by a lot of ISPs.
    - sumtechguy 4y ago
      It is an interesting tricky problem. As the DNS system was designed for caching and an 'honor system'. How do you allow caching so you do not completely overwhelm the root servers yet still prevent modification/interception of the contents of the messages? DoH sort of addresses it. But then you still have to trust that the message has not been altered/intercepted at the DoH server. Yet we also want the server to be able to do a replay of that message to another client. Also how to retrofit all of the existing code that uses DNS that uses this magic system. It is like we need a lightweight one way TLS session but with certificate caching and a second request/response type with a hash. But you have to watch out for cert forging then. That the browsers are reacting in this way says we have a failure at the DNS software level. Those projects do seem to be giving the security that is wanted. So we are starting to get some fragmentation. Which can not be good. Perhaps we need new record types to support this?
      - tptacek 4y ago
- adql 4y ago
  the middlebox could just pass traffic for the unknown TLDs tho and that detectin method would be in vain
a13o 4y ago
Google makes money when people use 8.8.8.8, so there's an even more sinister interpretation. Chrome has all it's users DDoS other DNS servers, to raise the table stakes on running one. Now your infrastructure needs to handle apparently a whole second internet of bogus traffic. Google, of course, has the resources to meet this higher bar.
plextoria 4y ago
Well, Google isn't the only abuser here. The networks that hijack DNS request should share some of the blame, no?
- dspillett 4y ago
  Chromium is imposing the extra load on the system, so yes it is the abuser in that sense. That they are doing it for what is a good reason for the app's users is immaterial to whether the effect is bad for the root DNS servers. If Karen takes my sandwich from the company fridge, so I take some of Jon's lunch, so I don't starve, I'm not innocent because Karen started it, I've created a situation where there are two arseholes instead of one. This isn't quite what is happening here as the root servers are effectively a public resource and stuff in the fridge is all private resources, but close enough to make the point. > The networks that hijack DNS request should share some of the blame They should have all the blame for deliberately breaking part of agreed protocols for their own gain. But that doesn't make anything we do in response to that right by virtue of us doing it because we have been wronged.
- tptacek 4y ago
  This isn't an "abuse". These are the Internet root DNS servers. We don't design applications to tiptoe around them any more than we tiptoe around the capacity of core routers. These is a huge amount of capacity and all of root DNS together is a rounding error on total Internet traffic.
- jeroenhd 4y ago
  They should, but Google's "solution" to the hijacking is the problem here. If you're going to hijack NXDOMAINs, you can just ignore requests to non-existent TLDs in your scheme and Chrome will be none the wiser. Google manages entire TLDs, surely they can use their own DNS servers for this purpose.
capableweb 4y ago
> What an arrogant hack. It's hard to believe Google engineers would find that a reasonable approach. Reminds me of the recent "Go module mirror fiasco" where Google found it fair to clone repositories at a rate of ~2,500 per hour in order to essentially proxy Go modules. - "Sourcehut will blacklist the Go module mirror" - https://news.ycombinator.com/item?id=34310674 https://news.ycombinator.com/item?id=34310674 After the drama become very much public, they finally decided to address the issue in a good way.
- tptacek 4y ago
  That doesn't at all seem like what happened (the changes were in the works prior to the drama), but this is a high-drama tangent unrelated to the story here. This isn't some tiny authoritative DNS server being flooded unexpectedly with queries. These are the Internet DNS root services. They have to keep up with this kind of traffic. It's their literal job description.
- adql 4y ago
  To be fair, that's less than one a second. But I dislike the whole GOPROXY design, like it breaking private repos by default and having to set some env variables to make this stupid tool download stuff from server I told it to download.
  - capableweb 4y ago
    > To be fair, that's less than one a second. Well, taking the context into consideration, I'd still say it's too much. Context being they were full git clones and the traffic ended up representing "70% of all outgoing network traffic from git.sr.ht".
- deleted 4y ago
  [deleted]
oaiey 4y ago
Well .. the root servers and DNS platform in general provide name resolution. Traditionally we think that as a feature/function name -> IP. But there non functional aspects also to consider. Distribution is one (DNS is really good on that), security (DNSSEC comes to mind), multi-transport channel (e.g. DNS over HTTP) are others but like seen in this case there is a non functional need, that interception can be detected. DNS does not deliver that (yet). But it is clearly a need of it in the DNS platform. So calling it abuse, is wrong. It is a dirty hack for a non-existing feature. It is technical debt of the DNS platform and the root server suffer for it because the ISPs and in-house DNS resolvers create the problem. Being angry at Google we can anyway be. They have enough money, enough people and enough power to either fix this financially or as a feature within the DNS platform.
deleted 4y ago
[deleted]
BenjiWiebe 4y ago
I've switched to 'ping 1.1' (1.0.0.1).

deleted 4y ago

[deleted]

deleted 4y ago
[deleted]

mschuster91 4y ago

The problem is, there is no better approach to check for intercepting middleboxes. Using a well-known path (like gstatic.com/generate_204) works for detecting if the user has a captive portal between them and the Internet, but not if the user's provider messes around with DNS.

blueflow 4y ago
Google could host their own DNS servers who only give out NXD and test against that.
- mschuster91 4y ago
  As soon as Google would do that, providers would implement exceptions.
  - adql 4y ago
    They could do DPI, allow traffic for unknown domains, intercept everything else. Differentiating random domain from "what user might actually type by accident" is also pretty trivial
inetknght 4y ago
> there is no better approach to check for intercepting middleboxes I can see some purposes for detecting middleboxes. I've done it. It usually doesn't involve DNS. It does involve certificate pinning though. > detecting if the user has a captive portal between them and the Internet That's easy. Try to browse to something. If succeeds but the certificate isn't valid then the user probably has a captive portal. That, or your pinned certificate has been revoked. > detecting ... if the user's provider messes around with DNS Certificate pinning, again, comes to the rescue. Pin a certificate to your own DoH server and then use DoH to look up whatever you need. If you can't connect to your DoH server then you effectively aren't (or shouldn't be) connected to the internet.
- opheliate 4y ago
  The kind of interception that is being discussed here is the interception of DNS queries which would return an NXDOMAIN result to instead point to an advertising page. This is done by a lot of ISPs.
  - sumtechguy 4y ago
    It is an interesting tricky problem. As the DNS system was designed for caching and an 'honor system'. How do you allow caching so you do not completely overwhelm the root servers yet still prevent modification/interception of the contents of the messages? DoH sort of addresses it. But then you still have to trust that the message has not been altered/intercepted at the DoH server. Yet we also want the server to be able to do a replay of that message to another client. Also how to retrofit all of the existing code that uses DNS that uses this magic system. It is like we need a lightweight one way TLS session but with certificate caching and a second request/response type with a hash. But you have to watch out for cert forging then. That the browsers are reacting in this way says we have a failure at the DNS software level. Those projects do seem to be giving the security that is wanted. So we are starting to get some fragmentation. Which can not be good. Perhaps we need new record types to support this?
    - tptacek 4y ago
adql 4y ago
the middlebox could just pass traffic for the unknown TLDs tho and that detectin method would be in vain

a13o 4y ago

Google makes money when people use 8.8.8.8, so there's an even more sinister interpretation. Chrome has all it's users DDoS other DNS servers, to raise the table stakes on running one. Now your infrastructure needs to handle apparently a whole second internet of bogus traffic. Google, of course, has the resources to meet this higher bar.

plextoria 4y ago

Well, Google isn't the only abuser here. The networks that hijack DNS request should share some of the blame, no?

dspillett 4y ago
Chromium is imposing the extra load on the system, so yes it is the abuser in that sense. That they are doing it for what is a good reason for the app's users is immaterial to whether the effect is bad for the root DNS servers. If Karen takes my sandwich from the company fridge, so I take some of Jon's lunch, so I don't starve, I'm not innocent because Karen started it, I've created a situation where there are two arseholes instead of one. This isn't quite what is happening here as the root servers are effectively a public resource and stuff in the fridge is all private resources, but close enough to make the point. > The networks that hijack DNS request should share some of the blame They should have all the blame for deliberately breaking part of agreed protocols for their own gain. But that doesn't make anything we do in response to that right by virtue of us doing it because we have been wronged.
tptacek 4y ago
This isn't an "abuse". These are the Internet root DNS servers. We don't design applications to tiptoe around them any more than we tiptoe around the capacity of core routers. These is a huge amount of capacity and all of root DNS together is a rounding error on total Internet traffic.
jeroenhd 4y ago
They should, but Google's "solution" to the hijacking is the problem here. If you're going to hijack NXDOMAINs, you can just ignore requests to non-existent TLDs in your scheme and Chrome will be none the wiser. Google manages entire TLDs, surely they can use their own DNS servers for this purpose.

capableweb 4y ago

> What an arrogant hack. It's hard to believe Google engineers would find that a reasonable approach. Reminds me of the recent "Go module mirror fiasco" where Google found it fair to clone repositories at a rate of ~2,500 per hour in order to essentially proxy Go modules. - "Sourcehut will blacklist the Go module mirror" - https://news.ycombinator.com/item?id=34310674 https://news.ycombinator.com/item?id=34310674 After the drama become very much public, they finally decided to address the issue in a good way.

tptacek 4y ago
That doesn't at all seem like what happened (the changes were in the works prior to the drama), but this is a high-drama tangent unrelated to the story here. This isn't some tiny authoritative DNS server being flooded unexpectedly with queries. These are the Internet DNS root services. They have to keep up with this kind of traffic. It's their literal job description.
adql 4y ago
To be fair, that's less than one a second. But I dislike the whole GOPROXY design, like it breaking private repos by default and having to set some env variables to make this stupid tool download stuff from server I told it to download.
- capableweb 4y ago
  > To be fair, that's less than one a second. Well, taking the context into consideration, I'd still say it's too much. Context being they were full git clones and the traffic ended up representing "70% of all outgoing network traffic from git.sr.ht".
deleted 4y ago
[deleted]

oaiey 4y ago

Well .. the root servers and DNS platform in general provide name resolution. Traditionally we think that as a feature/function name -> IP. But there non functional aspects also to consider. Distribution is one (DNS is really good on that), security (DNSSEC comes to mind), multi-transport channel (e.g. DNS over HTTP) are others but like seen in this case there is a non functional need, that interception can be detected. DNS does not deliver that (yet). But it is clearly a need of it in the DNS platform. So calling it abuse, is wrong. It is a dirty hack for a non-existing feature. It is technical debt of the DNS platform and the root server suffer for it because the ISPs and in-house DNS resolvers create the problem. Being angry at Google we can anyway be. They have enough money, enough people and enough power to either fix this financially or as a feature within the DNS platform.

deleted 4y ago

[deleted]

BenjiWiebe 4y ago

I've switched to 'ping 1.1' (1.0.0.1).