9 ms·

> and ISP provided DNS is almost universally terrible. They’ll ignore TTLs, rewrite NXDOMAIN responses into the IP address of their ad-laced web search, and occ

> and ISP provided DNS is almost universally terrible. They’ll ignore TTLs, rewrite NXDOMAIN responses into the IP address of their ad-laced web search, and occasionally just highjack every single query to send you to a page saying you’re approaching your bandwidth quota Universally? Perhaps in the US? But IoT devices are sold worldwide. While I do run my own recursive resolver I checked my ISP's and they're behaving fairly reasonable and do none of the above and I have a direct (contractual) relationship with them and we reside in the same jurisdiction so at least in principle I could apply pressure to them if they do something shady. The same can't be said about google or cloudflare. > Just be aware DNS over HTTPS is a thing now, and while the devices I’m responsible for aren’t going to try and evade your redirects, the companies that are trying to make sure ads get delivered will absolutely switch to DoH which will be much more difficult to work around. I'm sure google had only our best interests in mind when unleashing that on us.

acdha 6y ago
It’s not just the U.S. A problem report once lead me to discover that the Spanish company Telefónica cached DNS results for years longer than the TTL. My error monitoring also picked up content injection (prior to HTTPS) and DNS hijacking around the world. Before going with the conspiratorial take note that Mozilla shipped DoH early and both Microsoft and Apple implemented it. Untrustworthy ISPs are a real problem even if Google deservedly gets suspicion about their motives. This isn’t another AMP.
- syshum 6y ago
  At this point Mozilla is just the Non-profit Arm of Google, They do what google tells them.... and DoH is absolutely designed to get around network based security and filtering, both for Ad's and other reasons.
  - acdha 6y ago
    > At this point Mozilla is just the Non-profit Arm of Google, They do what google tells them You can just say you don’t follow this closely. Mozilla is not perfect but they do push for privacy, with an increasingly limited amount of negotiating power. > DoH is absolutely designed to get around network based security and filtering, both for Ad's and other reasons. This is similarly reflecting a poor understanding of the situation. DoH can’t get around network filtering - if you block packets, there’s no magic trick to bypass it. It’s great for preventing ISPs from tampering with traffic or monitoring activity (this will also require eSNI to complete) but it’s not giving an attacker any capability they didn’t already have. If you’re concerned about security you’re fooling yourself if you don’t have endpoint management and some level of network segmentation and egress control. Attackers have hard-coded DNS servers, C&C endpoints, etc. for decades.
    - fiddlerwoaroof 6y ago
      DoH absolutely can get network filtering: if I block port 53 outbound to control DNS queries within my network, DoH (and other tunneling technologies) is bypassing my network filtering of DNS.
      - shawnz 6y ago
        By the same logic HTTPS is bypassing your network filtering of every service which happens to have an alternative available over HTTPS. So do you block HTTPS?
        fiddlerwoaroof 6y ago
        I’ve been thinking about installing a proxy with my own root for this reason.
        deleted 6y ago
        [deleted]
        shawnz 6y ago
        I would not want to use your network if that was the case, and I wonder if you similarly would use someone else's network if it was configured like that. I don't think network operators who provide access to the Internet can realistically expect to control what their users do on the Internet, unless the network operator is also the administrator of those workstations.
        fiddlerwoaroof 6y ago
        Maybe, but my rule is: my home network, my rules. I can’t really do this invisibly anyways: any device I don’t control will get certificate validation errors.
        syshum 6y ago
        The problem here is even the device owner is not the "administrator" of the device as we have lost ownership rights, locked behind EULA's, Patents and copyright The point of the original story is "your" meaning a device you own, is ignoring your network controls it is highly unlikely that the TV manufacturer is going to allow me to install my own custom root certs to inspect their traffic to HTTPS, so yes DoH and other things are a threat to network security, because if the TV become compromised I have limited administrative controls to prevent it other the blocking it completely which is a poor response to the problem DoH is a solution in search of a problem that can be solves in better more user friendly ways
  - skinkestek 6y ago
    > At this point Mozilla is just the Non-profit Arm of Google, They do what google tells them.... As someone who has used (and still use Firefox) continously since around 2005 it surely feels that way sometimes and some of the decisions I see would make much more sense to me if I knew top management was somehow in Googles pocket. (That said 1. for my workflows I still consider Firefox the best browser. 2. switching would only make it even easier for Google. 3. I always hope something will change and Firefox will become really really great again or someone will fork it.
    - inetknght 6y ago
      > some of the decisions I see would make much more sense to me if I knew top management was somehow in Googles pocket. Consider. [0] https://en.wikipedia.org/wiki/Mozilla_Corporation#Google https://en.wikipedia.org/wiki/Mozilla_Corporation#Google [1] https://www.theverge.com/2020/8/15/21370020/mozilla-google-firefox-search-engine-browser https://www.theverge.com/2020/8/15/21370020/mozilla-google-f...
    - wolco2 6y ago
      They are in google pockets for 95% of revenue. But I think as a browser with a smaller market they join things and change things to be like everyone else. There was a recent change to facebook picture albums that lazily loads pictures as you scroll. In large albums this grinds to a halt after a few pages and each scroll takes 30 seconds to load the next set of images. Chrome handles this smoothly. For social media I had to switch to chrome. Firefox is still better for privacy and I use it when I can even though it feels like it's slowing down after each upgrade.
    - 153791098c 6y ago
      > 3. I always hope something will change and Firefox will become really really great again or someone will fork it. Abrowser (from trisquel) is what firefox should have been. Best fork out there imo.
  - stjohnswarts 6y ago
    Extraordinary claims require extraordinary proof. Of which I presume you have lots to share with the HN community?
- will_pseudonym 6y ago
  For anyone else not in the know, "DoH" means "DNS-over-HTTPS" Here's an article from November 19, 2020 about the rollout and some of the criticisms/backlash: https://www.zdnet.com/article/fearing-drama-mozilla-opens-public-consultation-before-worldwide-firefox-doh-rollout/ https://www.zdnet.com/article/fearing-drama-mozilla-opens-pu...
- topranks 6y ago
  Can you link to the results of your study?
neurostimulant 6y ago
> Universally? Perhaps in the US? But IoT devices are sold worldwide. Ugh, the situation is even worst in countries with censorship laws. For example, in my country, all ISP are required to intercept all DNS requests and filter all requests to any blocked domains found in the government block list. At least they're transparent about which sites are blocked though (the list is publicly available to query or download), but the fact that all DNS requests are intercepted causes various technical issues and some ISP are trying to profit from it by redirecting the blocked query to their own ads-laden landing pages. They even went as far as inspecting http host header as well as randomly injecting scripts on unencrypted http requests.
- loughnane 6y ago
  Curious which country is this? A public list of censored sites seems an odd thing to exist
  - garaetjjte 6y ago
    OP talks about somewhere else, but Poland have public registry of domain names that every ISP must block/redirect to Ministry of Finance website in their DNS resolver: https://hazard.mf.gov.pl/ https://hazard.mf.gov.pl/ (gambling sites running without proper license) (this is only about ISP resolver, no intercepting going on)
  - manquer 6y ago
    India definitely does this. It is usually porn and piracy etc related
  - neurostimulant 6y ago
    Indonesia. The official block list is available here: https://trustpositif.kominfo.go.id/ https://trustpositif.kominfo.go.id/
Spooky23 6y ago
ISPs will also enforce other things using DNS. For example when Spectrum was fighting with Netflix, they sent traffic to some overloaded peering connection. DNS over HTTPS is an abomination sold as snake oil security.
tzs 6y ago
> I'm sure google had only our best interests in mind when unleashing that on us. Mozilla did more to unleash it than Google. Someone from Mozilla coauthored the RFC for it, and Mozilla had browser support for it first.
c0nsumer 6y ago
The absolute worst version I had of this was in China (and I think Korea as well). I was troubleshooting an issue where a company's client computers which connected to the company VPN wouldn't have internet access post-connection. The problem turned out to be that instead of sending a NXDOMAIN they'd return their ad server and then _always_ send back some JS to show ads. The company network used PAC files (these are a piece of JS with a single FindProxyForURL() function) via an internal-only URI to steer most requests to our proxies, while keeping some internal. The problem came about when clients would first start up the OS would attempt to access the internal-only URI before the VPN client finished connecting. In a normal network it'd get nothing and life would carry on. With this problematic ISP they'd get something that /should/ have been a PAC file, but because it was some other piece of JS without FindProxyForURL() it wouldn't work as a PAC file and thus the client wouldn't go to the proxy. The expiration on this piece of JS was set to some absurd amount of time, so when the client would eventually try hitting our PAC file server again (happens every 20 minutes on Windows) it wouldn't get our file because it thought the garbage one from the ISP was newer. And the ISP updated their hijacking JS more frequently than we updated our PAC file. There were two possible solutions to this. One, routinely touch the PAC file so it's date was newer than whatever the ISP put out. Or two, set up an external A record for the internal PAC server name to keep the hijacking from working. We went with the second.
dastx 6y ago
My ISP in the UK returns NXDOMAIN for records pointing to 10. ips. I had to go through a lengthy process to have that issue fixed with them. Their support had no idea, and was trying to tell me it's nothing to do with them.
carstenhag 6y ago
In Germany, Telekom used to hijack nxdomain. A court actually found them guilty and since a half year or so they are no longer doing that.
- the8472 6y ago
  I think that actually supports my point. A customer has a relationship with the ISP and can actually sue them. If people start putting google or cloudflare DNS into lots of devices and apps and those start doing something questionable then you have no recourse and even if they did they'd have to go after dozens of different vendors to change their behavior instead of the single ISP.
  - frankchn 6y ago
    It doesn’t matter to the device OEM when the customer demands a refund because the device stopped working, even though the root cause is the ISP hijacking the DNS request and the device OEM has nothing to do with it. Easier to work around this apparently common ISP practice.
  - lmns 6y ago
    Deutsche Telekom hijacked NXDOMAINs for over 10 years, though (maybe even longer). It's not like they did it for a few months and then got sued.
inopinatus 6y ago
> I'm sure google had only our best interests in mind when unleashing that on us. Laughably, DoH is from the same gang of A-record squatters that refused to incorporate SRV into HTTP on the (now very evidently spurious) grounds that it could, in some scenarios, require an extra packet, and they couldn't work out how to make it backwards compatible. Having been comprehensively hijacked by the interests of advertising companies, my view of the HTTP WG has never been lower. And that's a shame because there are some smart people there, tasked with slowly eroding away the last semblance of end-to-end transparency.
kbumsik 6y ago
> Universally? Perhaps in the US? But IoT devices are sold worldwide. The ISP provided DNS in South Korea is absolutely terrible too. Setting 1.1.1.1 or 8.8.8.8 always serves far better experience than the ISP's.
tvararu 6y ago
As a counter-point, my UK ISP uses Cloudflare 1.1.1.1 last I checked.

acdha 6y ago

It’s not just the U.S. A problem report once lead me to discover that the Spanish company Telefónica cached DNS results for years longer than the TTL. My error monitoring also picked up content injection (prior to HTTPS) and DNS hijacking around the world. Before going with the conspiratorial take note that Mozilla shipped DoH early and both Microsoft and Apple implemented it. Untrustworthy ISPs are a real problem even if Google deservedly gets suspicion about their motives. This isn’t another AMP.

syshum 6y ago
At this point Mozilla is just the Non-profit Arm of Google, They do what google tells them.... and DoH is absolutely designed to get around network based security and filtering, both for Ad's and other reasons.
- acdha 6y ago
  > At this point Mozilla is just the Non-profit Arm of Google, They do what google tells them You can just say you don’t follow this closely. Mozilla is not perfect but they do push for privacy, with an increasingly limited amount of negotiating power. > DoH is absolutely designed to get around network based security and filtering, both for Ad's and other reasons. This is similarly reflecting a poor understanding of the situation. DoH can’t get around network filtering - if you block packets, there’s no magic trick to bypass it. It’s great for preventing ISPs from tampering with traffic or monitoring activity (this will also require eSNI to complete) but it’s not giving an attacker any capability they didn’t already have. If you’re concerned about security you’re fooling yourself if you don’t have endpoint management and some level of network segmentation and egress control. Attackers have hard-coded DNS servers, C&C endpoints, etc. for decades.
  - fiddlerwoaroof 6y ago
    DoH absolutely can get network filtering: if I block port 53 outbound to control DNS queries within my network, DoH (and other tunneling technologies) is bypassing my network filtering of DNS.
    - shawnz 6y ago
      By the same logic HTTPS is bypassing your network filtering of every service which happens to have an alternative available over HTTPS. So do you block HTTPS?
      - fiddlerwoaroof 6y ago
        I’ve been thinking about installing a proxy with my own root for this reason.
        deleted 6y ago
        [deleted]
        shawnz 6y ago
        I would not want to use your network if that was the case, and I wonder if you similarly would use someone else's network if it was configured like that. I don't think network operators who provide access to the Internet can realistically expect to control what their users do on the Internet, unless the network operator is also the administrator of those workstations.
        fiddlerwoaroof 6y ago
        Maybe, but my rule is: my home network, my rules. I can’t really do this invisibly anyways: any device I don’t control will get certificate validation errors.
        syshum 6y ago
        The problem here is even the device owner is not the "administrator" of the device as we have lost ownership rights, locked behind EULA's, Patents and copyright The point of the original story is "your" meaning a device you own, is ignoring your network controls it is highly unlikely that the TV manufacturer is going to allow me to install my own custom root certs to inspect their traffic to HTTPS, so yes DoH and other things are a threat to network security, because if the TV become compromised I have limited administrative controls to prevent it other the blocking it completely which is a poor response to the problem DoH is a solution in search of a problem that can be solves in better more user friendly ways
- skinkestek 6y ago
  > At this point Mozilla is just the Non-profit Arm of Google, They do what google tells them.... As someone who has used (and still use Firefox) continously since around 2005 it surely feels that way sometimes and some of the decisions I see would make much more sense to me if I knew top management was somehow in Googles pocket. (That said 1. for my workflows I still consider Firefox the best browser. 2. switching would only make it even easier for Google. 3. I always hope something will change and Firefox will become really really great again or someone will fork it.
  - inetknght 6y ago
    > some of the decisions I see would make much more sense to me if I knew top management was somehow in Googles pocket. Consider. [0] https://en.wikipedia.org/wiki/Mozilla_Corporation#Google https://en.wikipedia.org/wiki/Mozilla_Corporation#Google [1] https://www.theverge.com/2020/8/15/21370020/mozilla-google-firefox-search-engine-browser https://www.theverge.com/2020/8/15/21370020/mozilla-google-f...
  - wolco2 6y ago
    They are in google pockets for 95% of revenue. But I think as a browser with a smaller market they join things and change things to be like everyone else. There was a recent change to facebook picture albums that lazily loads pictures as you scroll. In large albums this grinds to a halt after a few pages and each scroll takes 30 seconds to load the next set of images. Chrome handles this smoothly. For social media I had to switch to chrome. Firefox is still better for privacy and I use it when I can even though it feels like it's slowing down after each upgrade.
  - 153791098c 6y ago
    > 3. I always hope something will change and Firefox will become really really great again or someone will fork it. Abrowser (from trisquel) is what firefox should have been. Best fork out there imo.
- stjohnswarts 6y ago
  Extraordinary claims require extraordinary proof. Of which I presume you have lots to share with the HN community?
will_pseudonym 6y ago
For anyone else not in the know, "DoH" means "DNS-over-HTTPS" Here's an article from November 19, 2020 about the rollout and some of the criticisms/backlash: https://www.zdnet.com/article/fearing-drama-mozilla-opens-public-consultation-before-worldwide-firefox-doh-rollout/ https://www.zdnet.com/article/fearing-drama-mozilla-opens-pu...
topranks 6y ago
Can you link to the results of your study?

neurostimulant 6y ago

> Universally? Perhaps in the US? But IoT devices are sold worldwide. Ugh, the situation is even worst in countries with censorship laws. For example, in my country, all ISP are required to intercept all DNS requests and filter all requests to any blocked domains found in the government block list. At least they're transparent about which sites are blocked though (the list is publicly available to query or download), but the fact that all DNS requests are intercepted causes various technical issues and some ISP are trying to profit from it by redirecting the blocked query to their own ads-laden landing pages. They even went as far as inspecting http host header as well as randomly injecting scripts on unencrypted http requests.

loughnane 6y ago
Curious which country is this? A public list of censored sites seems an odd thing to exist
- garaetjjte 6y ago
  OP talks about somewhere else, but Poland have public registry of domain names that every ISP must block/redirect to Ministry of Finance website in their DNS resolver: https://hazard.mf.gov.pl/ https://hazard.mf.gov.pl/ (gambling sites running without proper license) (this is only about ISP resolver, no intercepting going on)
- manquer 6y ago
  India definitely does this. It is usually porn and piracy etc related
- neurostimulant 6y ago
  Indonesia. The official block list is available here: https://trustpositif.kominfo.go.id/ https://trustpositif.kominfo.go.id/

Spooky23 6y ago

ISPs will also enforce other things using DNS. For example when Spectrum was fighting with Netflix, they sent traffic to some overloaded peering connection. DNS over HTTPS is an abomination sold as snake oil security.

tzs 6y ago

> I'm sure google had only our best interests in mind when unleashing that on us. Mozilla did more to unleash it than Google. Someone from Mozilla coauthored the RFC for it, and Mozilla had browser support for it first.

c0nsumer 6y ago

The absolute worst version I had of this was in China (and I think Korea as well). I was troubleshooting an issue where a company's client computers which connected to the company VPN wouldn't have internet access post-connection. The problem turned out to be that instead of sending a NXDOMAIN they'd return their ad server and then _always_ send back some JS to show ads. The company network used PAC files (these are a piece of JS with a single FindProxyForURL() function) via an internal-only URI to steer most requests to our proxies, while keeping some internal. The problem came about when clients would first start up the OS would attempt to access the internal-only URI before the VPN client finished connecting. In a normal network it'd get nothing and life would carry on. With this problematic ISP they'd get something that /should/ have been a PAC file, but because it was some other piece of JS without FindProxyForURL() it wouldn't work as a PAC file and thus the client wouldn't go to the proxy. The expiration on this piece of JS was set to some absurd amount of time, so when the client would eventually try hitting our PAC file server again (happens every 20 minutes on Windows) it wouldn't get our file because it thought the garbage one from the ISP was newer. And the ISP updated their hijacking JS more frequently than we updated our PAC file. There were two possible solutions to this. One, routinely touch the PAC file so it's date was newer than whatever the ISP put out. Or two, set up an external A record for the internal PAC server name to keep the hijacking from working. We went with the second.

dastx 6y ago

My ISP in the UK returns NXDOMAIN for records pointing to 10. ips. I had to go through a lengthy process to have that issue fixed with them. Their support had no idea, and was trying to tell me it's nothing to do with them.

carstenhag 6y ago

In Germany, Telekom used to hijack nxdomain. A court actually found them guilty and since a half year or so they are no longer doing that.

the8472 6y ago
I think that actually supports my point. A customer has a relationship with the ISP and can actually sue them. If people start putting google or cloudflare DNS into lots of devices and apps and those start doing something questionable then you have no recourse and even if they did they'd have to go after dozens of different vendors to change their behavior instead of the single ISP.
- frankchn 6y ago
  It doesn’t matter to the device OEM when the customer demands a refund because the device stopped working, even though the root cause is the ISP hijacking the DNS request and the device OEM has nothing to do with it. Easier to work around this apparently common ISP practice.
- lmns 6y ago
  Deutsche Telekom hijacked NXDOMAINs for over 10 years, though (maybe even longer). It's not like they did it for a few months and then got sued.

inopinatus 6y ago

> I'm sure google had only our best interests in mind when unleashing that on us. Laughably, DoH is from the same gang of A-record squatters that refused to incorporate SRV into HTTP on the (now very evidently spurious) grounds that it could, in some scenarios, require an extra packet, and they couldn't work out how to make it backwards compatible. Having been comprehensively hijacked by the interests of advertising companies, my view of the HTTP WG has never been lower. And that's a shame because there are some smart people there, tasked with slowly eroding away the last semblance of end-to-end transparency.

kbumsik 6y ago

> Universally? Perhaps in the US? But IoT devices are sold worldwide. The ISP provided DNS in South Korea is absolutely terrible too. Setting 1.1.1.1 or 8.8.8.8 always serves far better experience than the ISP's.

tvararu 6y ago

As a counter-point, my UK ISP uses Cloudflare 1.1.1.1 last I checked.