10 ms·

This isn't malice. AT&T has an internal IP they assigned to 1.1.1.1 because it was unused and they used it as an image caching proxy so it browsing the internet

This isn't malice. AT&T has an internal IP they assigned to 1.1.1.1 because it was unused and they used it as an image caching proxy so it browsing the internet would feel faster on early phones. I've seen it when I was reverse engineering on Android a while back.

masklinn 8y ago
So it's not just malice but doubly so: they used an IP they didn't have the rights to and they're now blocking proper users of it.
- jamespo 8y ago
  "Never attribute to malice that which is adequately explained by stupidity"
  - paulie_a 8y ago
    Except when it is AT&T. Then you can just assume it was more malicious than you even originally thought
    - jdironman 8y ago
      Equal parts malice and stupidity.
  - Aissen 8y ago
    It's stupidity, then malice as a cover-up.
    - rando444 8y ago
      No, that's not what malice means. Unless you're actually trying to say that AT&T has a grudge against Cloudflare and are only doing this to harm their company. This is something more like negligence or gross negligence.
      - arbitrage 8y ago
        > only doing this to harm their company That's not wholly unthinkable as far as AT&T is concerned. They're historically a bad player.
  - jjoonathan 8y ago
    Hanlon's razor is dangerous. "Playing stupid" and its adult cousin "plausible deniability" trivially exploit the razor under adversarial conditions. That said, we agree on the probable cause of this particular issue.
- Wheaties466 8y ago
  and theres nothing we can do about it.
  - deleted 8y ago
    [deleted]
  - aesh2Xa1 8y ago
    If you own a device that is supported by a custom ROM such as Lineage OS then you can flash that and not worry about this change. Otherwise, you can purchase a different device, preferably a Nexus/Pixel, or at least one that's unlocked. If that's impossible for you then, yes, you're stuck with AT&T's "best efforts."
- netsec_burn 8y ago
  They used an IP that was originally reserved for what reserved IP's are used for. Now that Cloudflare convinced 1.1.1.1 to be released, I'm sure AT&T wants service continuity and had to make this decision, which is well within their rights as an ISP. I dislike AT&T so if this was entirely opinion-based, I would be against them here. But this is a knee-jerk reaction to a well justified decision.
  - unethical_ban 8y ago
    It was not ever a reserved IP address in the proper sense. They should have used something from RFC1918.
    - juliangoldsmith 8y ago
      Any sane gateway would block any access to anything from RFC1918, and using a private IP address for a public service is a bad idea in general.
      - unethical_ban 8y ago
        Fair enough, if it was traversing public (non-ATT) networks without a tunnel.
        juliangoldsmith 8y ago
        It would be reasonable for AT&T equipment to block any private IP traffic outside a customer's private network by default. That's ignoring that AT&T's network is public, and it wouldn't make sense to use a private IP for a service they provide. Reversing that would likely require AT&T to make a firewall change to literally every piece of equipment they operate, and that's assuming that they don't use the blocks internally. That, and I can guarantee that some customer, somewhere, would be using whatever IP they chose.
  - qmarchi 8y ago
    Except that it wasn't classified as a private IP address. They should've use something like 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16. The 1.0.0.0/8 range was owned by IANA from _1981_ up until 2010, when it was transfered to APNIC. (The 2.0.0.0/8 range was also owned by IANA until 2010, thentransfered to RIPE NCC). If you want to get technical, use of the space could be construed as theft. As for the continuity issue, it was stated that it was an old device, so they have no responsibility to continue supporting it, and considering the age of the device in question, it may not be able to connect to the existing network.
- simias 8y ago
  Let's not act like using a "probably not in-use IPv4 but we can't really be sure" is a crime against humanity. If you're designing any kind of large scale system over the internet you end up hitting the problem sooner or later (like how some VPN solutions started using 5.x.y.z to be sure not to clash with LAN IPs for instance). The real solution of course would be to switch to IPv6 where any vendor can claim some private address anywhere without any realistic risk of collision but we all know that we're not ready for that yet. By their own admission CF receives a ridiculous amount of garbage traffic at this IP, it was not absurd for AT&T engineers in the past to thing "well, we need an IP that we can be reasonably sure nobody is going to use and is never going to conflict with anything on any network, 1.1.1.1 seems reasonable". Seeing everybody in this thread jumping into conspiracy theories instead of the much more likely configuration issue is a bit disappointing for a community that's supposed to understand technology.
  - arbitrage 8y ago
    > it was not absurd for AT&T engineers in the past to thing That is an utterly unreasonable conclusion. The same logic resulted in Y2K, which was generally a huge waste of time, money, and resources. The same logic has resulted in the anemic adoption of IPv6, which is NOT a correct solution, because it doesn't work properly for large swaths of the public. The correct answer always was, and will continue to be, to use internal routes for internal routing, and external routes for external. Clashes with your LAN? Too fucking bad. This sort of pushing out of externalizes onto the customer results in the same exact outcome anyway: customer gets screwed. The customer always gets screwed. Don't rationalize the incompetence of engineers who should know better, and corporate execs who don't give a fuck.
  - denkmoon 8y ago
    Would you feel the same way if AT&T started blasting static on "unused" frequencies they don't have the right to? Do that and the FCC will come down on you like a tonne of bricks, and I feel that is absolutely justified.
  - eeeeeeeeeeeee 8y ago
    It's not reasonable. We have RFCs for a reason, which define which subnets can be used for public use, and which can be used internally. This has been written down for a long time and anyone working at ATT that can make these kind of decisions should know better.
- nickysielicki 8y ago
  AT&T regularly assigns my phone an IP in 10/8, instead of using 100.64/16 as they should [1]. IIRC, they used to even have the gall to use 172.16/12, which is crazy when you consider the amount of corporate networks using those addresses. This caused issues where my phone would try to get on wifi, but the DHCPACK would be sent along on the existing interface rather than the one coming up. So the wifi icon was continually bouncing back and forth. My only solution was to go into airplane mode and bring the cellular down before bringing the wifi up. I don't think Android ever addressed this issue, and I had to switch around the entire subnet to avoid the conflicts. If I knew enough about how Android worked, I'd write a patch to have all android interfaces in their own linux netns, with the dhcp client exec'd in that netns, that way you'd never have to worry about this sort of conflict. [1]: https://tools.ietf.org/html/rfc6598 https://tools.ietf.org/html/rfc6598
  - isaack 8y ago
    A carrier deploying NAT might not be using CGNAT, and hence, in compliance with the spec, decide not to use the 100.64/16 space. One requirement of CGNAT, for instance, requires that the carrier's router be able to handle "address crashes". [1] [1] https://tools.ietf.org/html/rfc6598#section-4 https://tools.ietf.org/html/rfc6598#section-4
    - nickysielicki 8y ago
      I think it says that you can use it even if you're not doing CGNAT, provided that you are a service provider. > In particular, Shared Address Space can only be used in Service Provider networks OR on routing equipment that is able to do address translation across router interfaces when the addresses are identical on two different interfaces. (edit: no, very clearly, "Devices MUST be capable of performing address translation when identical Shared Address Space ranges are used on two different interfaces." ) Also, is it wrong to assume that cellular networks are able to handle "address crashes" due to the inherent centralization that comes as a result of having clients maintain the same IP (and same connections) as the device hops from tower to tower? Maybe I don't understand the topics at play here...
dumbneurologist 8y ago
then it's stupidity and malice. stupid to think using the IP was a good idea malice to break my device in order to paper over their stupidity. And from a telecom no less - of all people, they should know better.
ryanlol 8y ago
You are wrong, that is obviously not the cause.
mrpippy 8y ago
Was this on AT&T's wireless network or wireline DSL/fiber? This problem is about the wireline network/CPE.
- netsec_burn 8y ago
  Wireless
jlg23 8y ago
If not malice, it is incredibly bad engineering. Every IP address on a foreign, public network has to be considered as "in use".
- Phylter 8y ago
  This is actually the reason that 1.1.1.1 gets so much traffic. People just assume it's not in use and can be abused a bit. Once it's available on the internet then all that excess traffic that was going nowhere gets transferred there. Still, it looks more like malice since there are other addresses besides 1.1.1.1 that are also blocked.

masklinn 8y ago

So it's not just malice but doubly so: they used an IP they didn't have the rights to and they're now blocking proper users of it.

jamespo 8y ago
"Never attribute to malice that which is adequately explained by stupidity"
- paulie_a 8y ago
  Except when it is AT&T. Then you can just assume it was more malicious than you even originally thought
  - jdironman 8y ago
    Equal parts malice and stupidity.
- Aissen 8y ago
  It's stupidity, then malice as a cover-up.
  - rando444 8y ago
    No, that's not what malice means. Unless you're actually trying to say that AT&T has a grudge against Cloudflare and are only doing this to harm their company. This is something more like negligence or gross negligence.
    - arbitrage 8y ago
      > only doing this to harm their company That's not wholly unthinkable as far as AT&T is concerned. They're historically a bad player.
- jjoonathan 8y ago
  Hanlon's razor is dangerous. "Playing stupid" and its adult cousin "plausible deniability" trivially exploit the razor under adversarial conditions. That said, we agree on the probable cause of this particular issue.
Wheaties466 8y ago
and theres nothing we can do about it.
- deleted 8y ago
  [deleted]
- aesh2Xa1 8y ago
  If you own a device that is supported by a custom ROM such as Lineage OS then you can flash that and not worry about this change. Otherwise, you can purchase a different device, preferably a Nexus/Pixel, or at least one that's unlocked. If that's impossible for you then, yes, you're stuck with AT&T's "best efforts."
netsec_burn 8y ago
They used an IP that was originally reserved for what reserved IP's are used for. Now that Cloudflare convinced 1.1.1.1 to be released, I'm sure AT&T wants service continuity and had to make this decision, which is well within their rights as an ISP. I dislike AT&T so if this was entirely opinion-based, I would be against them here. But this is a knee-jerk reaction to a well justified decision.
- unethical_ban 8y ago
  It was not ever a reserved IP address in the proper sense. They should have used something from RFC1918.
  - juliangoldsmith 8y ago
    Any sane gateway would block any access to anything from RFC1918, and using a private IP address for a public service is a bad idea in general.
    - unethical_ban 8y ago
      Fair enough, if it was traversing public (non-ATT) networks without a tunnel.
      - juliangoldsmith 8y ago
        It would be reasonable for AT&T equipment to block any private IP traffic outside a customer's private network by default. That's ignoring that AT&T's network is public, and it wouldn't make sense to use a private IP for a service they provide. Reversing that would likely require AT&T to make a firewall change to literally every piece of equipment they operate, and that's assuming that they don't use the blocks internally. That, and I can guarantee that some customer, somewhere, would be using whatever IP they chose.
- qmarchi 8y ago
  Except that it wasn't classified as a private IP address. They should've use something like 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16. The 1.0.0.0/8 range was owned by IANA from _1981_ up until 2010, when it was transfered to APNIC. (The 2.0.0.0/8 range was also owned by IANA until 2010, thentransfered to RIPE NCC). If you want to get technical, use of the space could be construed as theft. As for the continuity issue, it was stated that it was an old device, so they have no responsibility to continue supporting it, and considering the age of the device in question, it may not be able to connect to the existing network.
simias 8y ago
Let's not act like using a "probably not in-use IPv4 but we can't really be sure" is a crime against humanity. If you're designing any kind of large scale system over the internet you end up hitting the problem sooner or later (like how some VPN solutions started using 5.x.y.z to be sure not to clash with LAN IPs for instance). The real solution of course would be to switch to IPv6 where any vendor can claim some private address anywhere without any realistic risk of collision but we all know that we're not ready for that yet. By their own admission CF receives a ridiculous amount of garbage traffic at this IP, it was not absurd for AT&T engineers in the past to thing "well, we need an IP that we can be reasonably sure nobody is going to use and is never going to conflict with anything on any network, 1.1.1.1 seems reasonable". Seeing everybody in this thread jumping into conspiracy theories instead of the much more likely configuration issue is a bit disappointing for a community that's supposed to understand technology.
- arbitrage 8y ago
  > it was not absurd for AT&T engineers in the past to thing That is an utterly unreasonable conclusion. The same logic resulted in Y2K, which was generally a huge waste of time, money, and resources. The same logic has resulted in the anemic adoption of IPv6, which is NOT a correct solution, because it doesn't work properly for large swaths of the public. The correct answer always was, and will continue to be, to use internal routes for internal routing, and external routes for external. Clashes with your LAN? Too fucking bad. This sort of pushing out of externalizes onto the customer results in the same exact outcome anyway: customer gets screwed. The customer always gets screwed. Don't rationalize the incompetence of engineers who should know better, and corporate execs who don't give a fuck.
- denkmoon 8y ago
  Would you feel the same way if AT&T started blasting static on "unused" frequencies they don't have the right to? Do that and the FCC will come down on you like a tonne of bricks, and I feel that is absolutely justified.
- eeeeeeeeeeeee 8y ago
  It's not reasonable. We have RFCs for a reason, which define which subnets can be used for public use, and which can be used internally. This has been written down for a long time and anyone working at ATT that can make these kind of decisions should know better.
nickysielicki 8y ago
AT&T regularly assigns my phone an IP in 10/8, instead of using 100.64/16 as they should [1]. IIRC, they used to even have the gall to use 172.16/12, which is crazy when you consider the amount of corporate networks using those addresses. This caused issues where my phone would try to get on wifi, but the DHCPACK would be sent along on the existing interface rather than the one coming up. So the wifi icon was continually bouncing back and forth. My only solution was to go into airplane mode and bring the cellular down before bringing the wifi up. I don't think Android ever addressed this issue, and I had to switch around the entire subnet to avoid the conflicts. If I knew enough about how Android worked, I'd write a patch to have all android interfaces in their own linux netns, with the dhcp client exec'd in that netns, that way you'd never have to worry about this sort of conflict. [1]: https://tools.ietf.org/html/rfc6598 https://tools.ietf.org/html/rfc6598
- isaack 8y ago
  A carrier deploying NAT might not be using CGNAT, and hence, in compliance with the spec, decide not to use the 100.64/16 space. One requirement of CGNAT, for instance, requires that the carrier's router be able to handle "address crashes". [1] [1] https://tools.ietf.org/html/rfc6598#section-4 https://tools.ietf.org/html/rfc6598#section-4
  - nickysielicki 8y ago
    I think it says that you can use it even if you're not doing CGNAT, provided that you are a service provider. > In particular, Shared Address Space can only be used in Service Provider networks OR on routing equipment that is able to do address translation across router interfaces when the addresses are identical on two different interfaces. (edit: no, very clearly, "Devices MUST be capable of performing address translation when identical Shared Address Space ranges are used on two different interfaces." ) Also, is it wrong to assume that cellular networks are able to handle "address crashes" due to the inherent centralization that comes as a result of having clients maintain the same IP (and same connections) as the device hops from tower to tower? Maybe I don't understand the topics at play here...

dumbneurologist 8y ago

then it's stupidity and malice. stupid to think using the IP was a good idea malice to break my device in order to paper over their stupidity. And from a telecom no less - of all people, they should know better.

ryanlol 8y ago

You are wrong, that is obviously not the cause.

mrpippy 8y ago

Was this on AT&T's wireless network or wireline DSL/fiber? This problem is about the wireline network/CPE.

netsec_burn 8y ago
Wireless

jlg23 8y ago

If not malice, it is incredibly bad engineering. Every IP address on a foreign, public network has to be considered as "in use".

Phylter 8y ago
This is actually the reason that 1.1.1.1 gets so much traffic. People just assume it's not in use and can be abused a bit. Once it's available on the internet then all that excess traffic that was going nowhere gets transferred there. Still, it looks more like malice since there are other addresses besides 1.1.1.1 that are also blocked.