7 ms·

Considering that Comcast sniffs, intercepts, and injects into HTTP web sites for their customer notification system(data cap overages and such) this just scream

by Washuu 6y ago

Considering that Comcast sniffs, intercepts, and injects into HTTP web sites for their customer notification system(data cap overages and such) this just screams suspicious to me even if it seems like it is meant to be a good announcement. I am not sure how I am supposed to trust that they will do the right thing for their customers.

tmikaeld 6y ago
Agreed, if it was cloudflare I would have at least given them the benefits of a doubt, but not Comcast.
- rydre 6y ago
  Cloudflare is harmful to independent CDN's. They hide the originating i.p. address (no other does it) to the nameserver in the name of "privacy" so you can only have as much granularity as the nearest cloudflare server to user (anti-competitive). That is unless you buy an ipv4 block (because everyone is still on ipv4) and set up anycast. For non 1.1.1.1 dns users I can just set up varnish and they'll be served via GEO ip lookup so the domain gets pointed to the nearest ip address. This is much cheaper. I'm not going to buy an ipv4 block just for cloudflare dns users. Their privacy claim is a lie because your webserver is going to be exposed to the end user i.p. address anyways after resolving from the nameserver.
  - DoctorOW 6y ago
    EDIT: I associated the name Cloudflare with the main product. I forgot the context of this being about DNS and therefore my comment is about Cloudflare CDN not 1.1.1.1 > Cloudflare is harmful to independent CDN's. Cloudflare is a competitor to CDNs. You don't need to use a CDN with Cloudflare, they proxy your content fully and do caching along the way as needed. > Their privacy claim is a lie because your webserver is going to be exposed to the end user i.p. address anyways after resolving from the nameserver. It's not a lie. Cloudflare is the nameserver, and the CDN. So after resolution the end user still has just a Cloudflare IP.
    - rydre 6y ago
      > It's not a lie. Cloudflare is the nameserver, and the CDN. So after resolution the end user still has just a Cloudflare IP. > In 2011 Google wrote an IETF draft to send Client IP information using the EDNS0 extension and this is usually called ‘edns-client-subnet’. As a DNS client, it means that a truncated version of your IP address will be added into the DNS request. The DNS server will use this truncated IP address to make a more informed decision in how it responds so that you can be connected to the most optimal server. This standard is promoted by the Faster Internet initiative and already adopted by some leading vendors. Because it is designed to keep privacy, the sender has the freedom to limit the client IP information. Instead of sending a full IP address, the DNS server is able to send partial information such as /24 only. For instance, if your IP address is 66.214.81.22, the DNS server will only expose the first three octets, so 66–214–81. Armed with the real IP address of the querying device, the DNS server can now come up with a much more accurate response. With this more intelligent routing, customers have a better Internet experience with lower latency and faster speeds. Best of all, this integration is being done using an open standard that is available for any company to integrate into their own platform. source: https://engineering.salesforce.com/why-is-edns-important-for-content-delivery-85f5690744ba https://engineering.salesforce.com/why-is-edns-important-for... Cloudflare 1.1.1.1 for consumers kills EDNS "edns-client-subnet" and instead offers the ip of the nearest cloudflare server to the user even if the website is not using cloudflare. This means your website can not ever serve content faster then cloudflare even if you could potentially be faster. This is the reason why many internet archives do not allow access to cloudflare client dns (1.1.1.1) users as a form of protest. BTW, Cloudflare allows you to get informed about the end user's ip via a "x-forwarded-for" header.
      - deleted 6y ago
        [deleted]
      - presumably 6y ago
        There is only a single "archive" that does not allow access to Cloudflare DNS users - not many. It is also exceedingly unlikely that you have greater density of anycast PoPs than Cloudflare's 200+. In your case, you have zero...
  - tssva 6y ago
    Not forwarding EDNS client subnet is a requirement for Mozilla TRR partners. NextDNS also doesn't forward EDNS subnet client since it is a partner and soon Comcast will be joining that list. Although not currently a member of the program Quad9 also by default doesn't forward EDNS subnet info.
    - rydre 6y ago
      I host my name server's myself. If Mozilla is really doing this, they're misguided. All this does is make it impossible to serve requests from nearest webserver. You are getting the ip address anyways, so why do this? This means if I get someone from netherlands I'd have to redirect their requests from www.example.com to nl.example.com or buy an ipv4 block, set up anycast and then serve from the closest ipaddress/server. The end result is same. I'll always get the end user's ip address unless they use a VPN or something. This is a stupid decision by Mozilla. Too bad Firefox users when visiting websites making their own CDN's without anycast/country level redirects will see much slower sites.
      - presumably 6y ago
        What you're claiming is false. Cloudflare has over 200 PoPs; in your own name servers, you can use the Cloudflare Resolver's IP (which will be a "close to the user" IP, not 1.1.1.1) to do geotargeting and serve from your closest IP address/server.
        rydre 6y ago
        >What you're claiming is false. Cloudflare has over 200 PoPs; in your own name servers, you can use the Cloudflare Resolver's IP (which will be a "close to the user" IP, not 1.1.1.1) to do geotargeting and serve from your closest IP address/server. What if my server is closer than cloudflare? Why is cloudflare artificially limiting?
        willcipriano 6y ago
        I use cloudflare precisely because I don't want clients hitting the server directly. That's its entire purpose. For both caching and anti-ddos reasons.
sandworm101 6y ago
There is nothing that Comcast can do that would increase my opinion of them re privacy. In my security regime ISPs like them are on the other side. Last-mile ISPs are unsecured public networks that shouldn't be trusted any more than free airport wifi. I want them blind to everything I (and my client) does online. Encrypt everything. Route DNS to trusted non-profit entities. Serve me the encrypted data I request but otherwise I don't want to even have a conversation with Comcast.
- Skunkleton 6y ago
  > Route DNS to trusted non-profit entities. I'm sure you know this, but some readers might not. DNS is totally insecure. Even if you change your DNS server from the default to 1.1.1.1 or whatever, your ISP can and does still read and/or intercept these requests. This sort of interference is absolutely trivial to implement, even at scale. Don't think it isn't happening to you.
  - solarkraft 6y ago
    ... which is exactly why DoH is gaining attention. But I keep wondering: Can't the ISP trivially correlate the accessed IP addresses with their corresponding sites even without DNS query data?
    - SAI_Peregrinus 6y ago
      Only for sites with dedicated IPs. If they're hosted on some sort of cloud service then the ISP has to sniff the SNI data. And with ESNI coming to encrypt it that hole will be plugged soon.
      - the8472 6y ago
        That just means moving from the ISP in a prime position for snooping to various CDNs being in that prime position. You traded one master for another.
        6y ago
NelsonMinar 6y ago
I'm very suspicious of Comcast too. They've had hostile policies in their Internet management for as long as I can remember, going back to the days they'd forge RST packets because they didn't like customers using BitTorrent. OTOH as the article says, 'Joining Mozilla's program means that Comcast agreed that it won't "retain, sell, or transfer to any third party (except as may be required by law) any personal information, IP addresses, or other user identifiers, or user query patterns from the DNS queries sent from the Firefox browser'. I assume Mozilla will audit and keep Comcast honest here, or at least try. I just am left wondering what loophole Comcast has found.
- csharptwdec19 6y ago
  >I assume Mozilla will audit and keep Comcast honest here, or at least try. I just am left wondering what loophole Comcast has found. HAHAHAHAHAHA OH YOU ARE FUNNY. Sorry. Comcast will use the same loophole they use with everyone else. Kafkaesque policies aren't just for their customers, they exist throughout the organization. Took me 6 months to get a Secure VPN link that -they- insisted we use to set up their files. And 3 months of that was just getting the password reset. The account was locked because it wasn't being used (first 3 months was them fighting with firewall configs)... But then we were in this loop of they would reset the password, but the person who had to TELL us the password was reset would take so long to do so that it expired again by the time we got it. So, extrapolating to here, my guess is that any sort of audit will take forever to get data, probably incomplete sets... you get the picture.
proverbialbunny 6y ago
I hit commit on that feature. Sorry. It's not just Comcast. Every ISP that uses Akamai's software to power their ISP has this ability and possibly uses it, just not in obvious ways. And given that almost every ISP in the US, let alone the world, uses this software, well.. that's just how it is. Though, it's http only. You can always switch to https and they can not do anything. There is no key injection or anything going on thankfully. Also, encrypted DNS should make it moot as well.
- sprayk 6y ago
  How would encrypting DNS help me avoid Comcast MITMing my HTTP traffic to inject bandwidth cap notifications? Doesn't the system just inject a script tag into the appropriate place in the HTTP response?
  - entropicdrifter 6y ago
    HTTPS Everywhere + encrypted DNS blocks a huge chunk of what they can see without expending effort on you in particular
    - sprayk 6y ago
      that is not what I was asking. GP claimed that encrypted DNS would stop comcast from injecting notifications into HTTP traffic, I want to know how that would work, in the hopes that my assumptions about the system are wrong.
      - proverbialbunny 6y ago
        You make a good point. I worked on the code before encrypted DNS was a thing (or anything I knew about) so I'm going off of theory, not first hand experience. When a request is sent for a http web page it is ran through the layer 4 proxy. In there is a user profile where http injection can occur. It works by injecting JavaScript into the end of the web page. If the dns request is encrypted all of the handshaking goes through tls bypassing the proxy's view of this data for everything except disconnected http body data. However, it could be that as years have gone by it's been updated to take in http data without any sort of head and then it would work again. It's probably as simple as running some regex looking for </html>. So, me in my half awake state this morning didn't really think it through. In previous versions of the software this wouldn't be supported, but in hindsight it's not a terribly difficult problem to fix, so Comcast probably does support HTTP injection even when using encrypted DNS by now. My apologies.
chaostheory 6y ago
I also wonder what the RIAA and MPAA think of this? I know using a browser isn't ideal for piracy, but at the same time those two organizations are one of several major reasons why ISPs snoop on their customers.
- sprayk 6y ago
  does RIAA/MPAA snooping rely on DNS traffic at all? I was under the impression that the only thing RIAA/MPAA caught people for was the act of sharing, usually in the form of torrent seeding, and this was done by third parties they contracted out to who would provide a list of IPs they caught seeding. Then the list of IPs were resolved to ISPs who were sent subpoenas where they thought they had a good chance of ISP cooperation. Is there some legal, warrantless wiretapping I'm not aware of?
  - chaostheory 6y ago
    Usually they primarily go after torrents, but some websites have caught their ire over the years.
threw23434 6y ago
> Considering that Comcast sniffs, intercepts, and injects into HTTP web sites You think that is bad ? If you get a connection from India's 'premier' public telco BSNL, you'll be treated to ad-injections for random malware straight into your HTTP page. "Your govt. welcomes your appreciation for its 'top-notch' services."
MINIMAN10000 6y ago
I don't care what their intent is. Interception and injection from a third party is bad. With technology there are many ways to achieve the goal of notifying a customer without foul play.

by Washuu 6y ago

tmikaeld 6y ago

Agreed, if it was cloudflare I would have at least given them the benefits of a doubt, but not Comcast.

rydre 6y ago
Cloudflare is harmful to independent CDN's. They hide the originating i.p. address (no other does it) to the nameserver in the name of "privacy" so you can only have as much granularity as the nearest cloudflare server to user (anti-competitive). That is unless you buy an ipv4 block (because everyone is still on ipv4) and set up anycast. For non 1.1.1.1 dns users I can just set up varnish and they'll be served via GEO ip lookup so the domain gets pointed to the nearest ip address. This is much cheaper. I'm not going to buy an ipv4 block just for cloudflare dns users. Their privacy claim is a lie because your webserver is going to be exposed to the end user i.p. address anyways after resolving from the nameserver.
- DoctorOW 6y ago
  EDIT: I associated the name Cloudflare with the main product. I forgot the context of this being about DNS and therefore my comment is about Cloudflare CDN not 1.1.1.1 > Cloudflare is harmful to independent CDN's. Cloudflare is a competitor to CDNs. You don't need to use a CDN with Cloudflare, they proxy your content fully and do caching along the way as needed. > Their privacy claim is a lie because your webserver is going to be exposed to the end user i.p. address anyways after resolving from the nameserver. It's not a lie. Cloudflare is the nameserver, and the CDN. So after resolution the end user still has just a Cloudflare IP.
  - rydre 6y ago
    > It's not a lie. Cloudflare is the nameserver, and the CDN. So after resolution the end user still has just a Cloudflare IP. > In 2011 Google wrote an IETF draft to send Client IP information using the EDNS0 extension and this is usually called ‘edns-client-subnet’. As a DNS client, it means that a truncated version of your IP address will be added into the DNS request. The DNS server will use this truncated IP address to make a more informed decision in how it responds so that you can be connected to the most optimal server. This standard is promoted by the Faster Internet initiative and already adopted by some leading vendors. Because it is designed to keep privacy, the sender has the freedom to limit the client IP information. Instead of sending a full IP address, the DNS server is able to send partial information such as /24 only. For instance, if your IP address is 66.214.81.22, the DNS server will only expose the first three octets, so 66–214–81. Armed with the real IP address of the querying device, the DNS server can now come up with a much more accurate response. With this more intelligent routing, customers have a better Internet experience with lower latency and faster speeds. Best of all, this integration is being done using an open standard that is available for any company to integrate into their own platform. source: https://engineering.salesforce.com/why-is-edns-important-for-content-delivery-85f5690744ba https://engineering.salesforce.com/why-is-edns-important-for... Cloudflare 1.1.1.1 for consumers kills EDNS "edns-client-subnet" and instead offers the ip of the nearest cloudflare server to the user even if the website is not using cloudflare. This means your website can not ever serve content faster then cloudflare even if you could potentially be faster. This is the reason why many internet archives do not allow access to cloudflare client dns (1.1.1.1) users as a form of protest. BTW, Cloudflare allows you to get informed about the end user's ip via a "x-forwarded-for" header.
    - deleted 6y ago
      [deleted]
    - presumably 6y ago
      There is only a single "archive" that does not allow access to Cloudflare DNS users - not many. It is also exceedingly unlikely that you have greater density of anycast PoPs than Cloudflare's 200+. In your case, you have zero...
- tssva 6y ago
  Not forwarding EDNS client subnet is a requirement for Mozilla TRR partners. NextDNS also doesn't forward EDNS subnet client since it is a partner and soon Comcast will be joining that list. Although not currently a member of the program Quad9 also by default doesn't forward EDNS subnet info.
  - rydre 6y ago
    I host my name server's myself. If Mozilla is really doing this, they're misguided. All this does is make it impossible to serve requests from nearest webserver. You are getting the ip address anyways, so why do this? This means if I get someone from netherlands I'd have to redirect their requests from www.example.com to nl.example.com or buy an ipv4 block, set up anycast and then serve from the closest ipaddress/server. The end result is same. I'll always get the end user's ip address unless they use a VPN or something. This is a stupid decision by Mozilla. Too bad Firefox users when visiting websites making their own CDN's without anycast/country level redirects will see much slower sites.
    - presumably 6y ago
      What you're claiming is false. Cloudflare has over 200 PoPs; in your own name servers, you can use the Cloudflare Resolver's IP (which will be a "close to the user" IP, not 1.1.1.1) to do geotargeting and serve from your closest IP address/server.
      - rydre 6y ago
        >What you're claiming is false. Cloudflare has over 200 PoPs; in your own name servers, you can use the Cloudflare Resolver's IP (which will be a "close to the user" IP, not 1.1.1.1) to do geotargeting and serve from your closest IP address/server. What if my server is closer than cloudflare? Why is cloudflare artificially limiting?
        willcipriano 6y ago
        I use cloudflare precisely because I don't want clients hitting the server directly. That's its entire purpose. For both caching and anti-ddos reasons.

sandworm101 6y ago

There is nothing that Comcast can do that would increase my opinion of them re privacy. In my security regime ISPs like them are on the other side. Last-mile ISPs are unsecured public networks that shouldn't be trusted any more than free airport wifi. I want them blind to everything I (and my client) does online. Encrypt everything. Route DNS to trusted non-profit entities. Serve me the encrypted data I request but otherwise I don't want to even have a conversation with Comcast.

Skunkleton 6y ago
> Route DNS to trusted non-profit entities. I'm sure you know this, but some readers might not. DNS is totally insecure. Even if you change your DNS server from the default to 1.1.1.1 or whatever, your ISP can and does still read and/or intercept these requests. This sort of interference is absolutely trivial to implement, even at scale. Don't think it isn't happening to you.
- solarkraft 6y ago
  ... which is exactly why DoH is gaining attention. But I keep wondering: Can't the ISP trivially correlate the accessed IP addresses with their corresponding sites even without DNS query data?
  - SAI_Peregrinus 6y ago
    Only for sites with dedicated IPs. If they're hosted on some sort of cloud service then the ISP has to sniff the SNI data. And with ESNI coming to encrypt it that hole will be plugged soon.
    - the8472 6y ago
      That just means moving from the ISP in a prime position for snooping to various CDNs being in that prime position. You traded one master for another.
      - 6y ago

NelsonMinar 6y ago

I'm very suspicious of Comcast too. They've had hostile policies in their Internet management for as long as I can remember, going back to the days they'd forge RST packets because they didn't like customers using BitTorrent. OTOH as the article says, 'Joining Mozilla's program means that Comcast agreed that it won't "retain, sell, or transfer to any third party (except as may be required by law) any personal information, IP addresses, or other user identifiers, or user query patterns from the DNS queries sent from the Firefox browser'. I assume Mozilla will audit and keep Comcast honest here, or at least try. I just am left wondering what loophole Comcast has found.

csharptwdec19 6y ago
>I assume Mozilla will audit and keep Comcast honest here, or at least try. I just am left wondering what loophole Comcast has found. HAHAHAHAHAHA OH YOU ARE FUNNY. Sorry. Comcast will use the same loophole they use with everyone else. Kafkaesque policies aren't just for their customers, they exist throughout the organization. Took me 6 months to get a Secure VPN link that -they- insisted we use to set up their files. And 3 months of that was just getting the password reset. The account was locked because it wasn't being used (first 3 months was them fighting with firewall configs)... But then we were in this loop of they would reset the password, but the person who had to TELL us the password was reset would take so long to do so that it expired again by the time we got it. So, extrapolating to here, my guess is that any sort of audit will take forever to get data, probably incomplete sets... you get the picture.

proverbialbunny 6y ago

I hit commit on that feature. Sorry. It's not just Comcast. Every ISP that uses Akamai's software to power their ISP has this ability and possibly uses it, just not in obvious ways. And given that almost every ISP in the US, let alone the world, uses this software, well.. that's just how it is. Though, it's http only. You can always switch to https and they can not do anything. There is no key injection or anything going on thankfully. Also, encrypted DNS should make it moot as well.

sprayk 6y ago
How would encrypting DNS help me avoid Comcast MITMing my HTTP traffic to inject bandwidth cap notifications? Doesn't the system just inject a script tag into the appropriate place in the HTTP response?
- entropicdrifter 6y ago
  HTTPS Everywhere + encrypted DNS blocks a huge chunk of what they can see without expending effort on you in particular
  - sprayk 6y ago
    that is not what I was asking. GP claimed that encrypted DNS would stop comcast from injecting notifications into HTTP traffic, I want to know how that would work, in the hopes that my assumptions about the system are wrong.
    - proverbialbunny 6y ago
      You make a good point. I worked on the code before encrypted DNS was a thing (or anything I knew about) so I'm going off of theory, not first hand experience. When a request is sent for a http web page it is ran through the layer 4 proxy. In there is a user profile where http injection can occur. It works by injecting JavaScript into the end of the web page. If the dns request is encrypted all of the handshaking goes through tls bypassing the proxy's view of this data for everything except disconnected http body data. However, it could be that as years have gone by it's been updated to take in http data without any sort of head and then it would work again. It's probably as simple as running some regex looking for </html>. So, me in my half awake state this morning didn't really think it through. In previous versions of the software this wouldn't be supported, but in hindsight it's not a terribly difficult problem to fix, so Comcast probably does support HTTP injection even when using encrypted DNS by now. My apologies.

chaostheory 6y ago

I also wonder what the RIAA and MPAA think of this? I know using a browser isn't ideal for piracy, but at the same time those two organizations are one of several major reasons why ISPs snoop on their customers.

sprayk 6y ago
does RIAA/MPAA snooping rely on DNS traffic at all? I was under the impression that the only thing RIAA/MPAA caught people for was the act of sharing, usually in the form of torrent seeding, and this was done by third parties they contracted out to who would provide a list of IPs they caught seeding. Then the list of IPs were resolved to ISPs who were sent subpoenas where they thought they had a good chance of ISP cooperation. Is there some legal, warrantless wiretapping I'm not aware of?
- chaostheory 6y ago
  Usually they primarily go after torrents, but some websites have caught their ire over the years.

threw23434 6y ago

> Considering that Comcast sniffs, intercepts, and injects into HTTP web sites You think that is bad ? If you get a connection from India's 'premier' public telco BSNL, you'll be treated to ad-injections for random malware straight into your HTTP page. "Your govt. welcomes your appreciation for its 'top-notch' services."

MINIMAN10000 6y ago

I don't care what their intent is. Interception and injection from a third party is bad. With technology there are many ways to achieve the goal of notifying a customer without foul play.