9 ms·
AT&T stealing/misrouting 1.1.1.0/29 for some of their residential customers
- wemdyjreichert 9y agoCan confirm. Have that same router; same issue. 1.0.0.1 works fine, though.
- chx 9y agoWhat is truly disturbing here... who wrote these firmwares? You'd think something as widely distributed as an Internet gateway for a major ISP would be written by someone who at least know some Internet basics (like which IP ranges are public and which are private). And, how do you test if this can happen. And if they made this mistake then what else is lurking in that particular piece of software?
- voltagex_ 9y agoWait til you find out what Cisco has been doing for years http://www.revolutionwifi.net/revolutionwifi/2011/03/explaining-dhcp-server-1111.html http://www.revolutionwifi.net/revolutionwifi/2011/03/explain...
- chx 9y ago"The WLC intercepts client DHCP discovery packets, inserts it's own IP address configured on the egress interface into the relay agent field, and unicasts the DHCP packet to the configured DHCP server." Mess with DHCP packets, that's a paddlin'.
- pathseeker 9y agoThat's DHCP relay...
- noselasd 9y agoIt might be a config issue, not a firmware issue. There are numerous guides/howtos floating around for configuring cisco/juniper/etc. equipment that uses 1.1.1.1,2.2.2.2 for things like loopback addresses(these are used for very different things than the lo device of 127.0.0.1), or uses such addresses for peer to peer links etc. People just copy paste these configs without thinking, I'm not at all surprised 1.1.1.1 breaks left and right.
- zentiggr 9y agoWait, you weren't aware that most equipment manufacturers' software teams leave every edge case and design decision to a coin toss between 'fastest to code' and 'least effort to code' and let the rest of us bitch and scream about the stupidest of egregious bs for years until that piece of gear isn't sold anymore, then do it again with the next model?
- MertsA 9y agoSigh, those AT&T gateways are such a dumpster fire. What's sad is that it's not likely that there will be an easy way to fix this as those gateways do not have any kind of bridge mode which is a crime in and of itself. The best they've got is DMZ+ mode which fakes giving the LAN device the public IP address but it's still doing NAT behind the scenes and routing everything as normal so it doesn't work for any IP protocol other than TCP, UDP, and ICMP so no IPSEC tunnels or GRE with those modems.
- toast0 9y agoThe biggest crime is that AT&T uses these on their FTTH to turn ethernet from the GPON transceiver into ethernet for the home. Thankfully, in that application, you can bypass it, but it's still a tragedy. My equipment is perfectly capable of handling Ethernet, thanks (it's also perfectly capable of bridging 802.1x packets to the gateway, and keeping all the rest of the packets :)
- benjaminl 9y agoDo you have a write-up of this anywhere? When I installed AT&T fiber I looked into this briefly, but I didn't find any success stories other than manual VLAN reassignment.
- boshaus 9y agohttps://bzsparks.com/2016/10/05/using-an-ubiquiti-edgerouter-with-att-gigapower-fiber/ https://bzsparks.com/2016/10/05/using-an-ubiquiti-edgerouter... here's an example for a ubiquiti edgerouter to not have to do the vlan garbage. I'm going to try getting it to work on pfsense later this week.
- toast0 9y agoI was planning on cleaning up some hardcoded values so I could push it to some git service, but haven't gotten around to it. However, I'm not above a code dump. My IP is hard coded, but then again, I'm going to link you to my website hosted on that IP, so it is what it is. :) http://ruka.org/~toast/steal/ http://ruka.org/~toast/steal/ (note the README which is going to tell you much of the same thing below in different words) Here's the scoop. This is a FreeBSD program which opens up netgraph sockets against two physical ethernet interfaces and one virtual one. The physical interfaces are a) one connected to the ONT, b) one connected to the WAN port on the Residential GateWay. The virtual interface (ngeth0) is the one that you configure as your internet connection for FreeBSD (These interfaces are hardcoded in init_netgraph in steal_util.c; sorry!) Everytime a packet comes in from the ONT, RGW, or FreeBSD, there's a decision for what to do with the packet. Mostly the RGW and FreeBSD packets go to the ONT, but through the libalias NAT engine for connection tracking. Packets inbound from the ONT are checked in the NAT tables to see where they should go... if there's no clear place to send it, then it goes to the FreeBSD. That's mostly in steal_util:loop. I haven't figured out how to setup the virtual interface in the program, so see the bottom of README for how i do that. steal2.c is a little bit insane, it's basically setting up hot code loading. It's annoying to restart the daemon because the NAT tables are dropped, and I do a lot of Erlang work in my day job, so I really wanted to be able to hotload some things with this. So it uses dlopen and dlfunc to load steal_util.so and calls the init and loop functions. If init fails on a reload, it continues to use the old loop. There's some provision for updating the state data structure in a safe way. Theoretically, this might work in Linux too, I think netgraph got ported there a couple of times, but I don't know if libalias did. Anyway, the main ideas could certainly be ported into whatever tool is available; namely: proxy 802.1x to the RGW, drop EAPOL logoff packets; oversend ARP packets, spoof mac address of the RGW, make pcaps of packets you don't know how to handle, etc. :) Also of note: the ONT/RGW send packets with VLAN 0, FreeBSD strips this when reading packets via netgraph, and I haven't found a way to send packets with VLAN 0, which hasn't been a problem (yet?). If it becomes a problem, I suspect I'd need to switch to the packet apis that tcpdump uses (BPF), I think there may be a sending interface there too. And I'm also sadly moving out of AT&T fiber's reach this summer, so at that point, I won't be able to test anything else.
- aarongolliver 9y agoMy CenturyLink provided gateway gobbles up 1.1.1.1 as well, thankfully 1.0.0.1 works.
- CaliforniaKarl 9y agoI think Icomera’s bus WiFi systems also use 1.1.1.1 somewhere in the hotspot process. TBH, I’m not surprised, and this definitely isn’t the first time something like this has happened. For example, Hamachi (before being bought by LogMeIn) squatted on 5.0.0.0/8, and then (when 5.0.0.0/8 got allocated) on 25.0.0.0/8 (which was already allocated to the British MoD). This is really nothing new, and now that I think of it, not really surprising!
- buzer 9y agoIt's probably running Cisco WLC. https://www.reddit.com/r/networking/comments/88qt5k/announcing_1111_the_fastest_privacyfirst_consumer/dwmlcst/ https://www.reddit.com/r/networking/comments/88qt5k/announci...
- edwhitesell 9y ago1.1.1.1 is used by a LOT of Wifi authentication systems, and has been going back to 2000 or so.
- nodesocket 9y agoI just got bite by this. If your ISP is AT&T (I have AT&T fiber) don't use 1.1.1.1, only use the CloudFlare secondary DNS resolver 1.0.0.1. <<>> DiG 9.10.6 <<>> google.com @1.1.1.1 ;; global options: +cmd ;; connection timed out; no servers could be reached
- saas_co_de 9y agodoes the term "stealing" really apply here?
- colejohnson66 9y agoIt is a bit hyperbolic
- gcb0 9y agoyes, unless ip allocation is completely free of direct and indirect costs.
- sprayk 9y agoyes. At the very least, there are the autonomous system registration fees paid to ARIN for the prefix. CloudFlare might just be paying APNIC for the two addresses it needs, though, or in some kind of agreement. Either way, CloudFlare paid in some way to be the sole entity on the internet allowed to advertise for prefixes containing 1.0.0.1 and 1.1.1.1 over BGP to its peers.
- pathseeker 9y agoWell AT&T isn't advertising those prefixes to other networks though. This is more like censorship than stealing.
- dragonwriter 9y agoYes, but without net neutrality it's perfectly legal theft.
- aardvark7 9y agoyes
- darklajid 9y agoI'd say 'abusing' is the term I'd go for - and it seems awkward for an ISP/network operator of that size to ignore the rules. If mum and dad's home network uses 1.1.1.1 because it's "cute" or whatever, that's understandable if still wrong. But why do network providers that employ lots of people that should know better do this? I cannot believe that it wasn't brought up, ever ("Hey, you know.. We should use 10.0.0.0/8 maybe?"). I gotta assume that someone in charge shrugged and said "Who cares?".
- jlgaddis 9y agoYep, I noticed this yesterday and mentioned it [0] when trying out Cloudflare's new public DNS service. A Cloudflare employee (marty) also mentioned on NANOG that they were aware of several other devices with the same issue. [0]: https://news.ycombinator.com/item?id=16729845 https://news.ycombinator.com/item?id=16729845
- RKearney 9y agoComcast / xfinity was null routing 1.1.1.1/32 for residential customers up until Monday morning/afternoon. They removed the null route fairly quickly. I assume it was done to reduce the amount of garbage traffic going to 1.1.1.1 from hitting Comcast's core.
- raverbashing 9y agoWhich garbage traffic? Where does it come from?
- matthew-wegner 9y agoPart of CloudFlare’s arrangement with APNIC is to study this traffic. See “Enter 1.1.1.1” section on this post: https://blog.cloudflare.com/announcing-1111/ https://blog.cloudflare.com/announcing-1111/
- cpeterso 9y agoSo is all this 1.1.1.1 garbage traffic now directed to Cloudflare's servers? Or will ISPs port filter everything sent to 1.1.1.1 except DNS (and DNS-over-HTTPS) requests headed for Cloudflare?
- Arnt 9y agoCan't really filter out such things efficiently, so Cloudflare gets it, yes. https://www.nanog.org/sites/default/files/wed.general.trafficdiversion.serodio.10.pdf https://www.nanog.org/sites/default/files/wed.general.traffi...
- PhantomGremlin 9y agoIs there still some sort of traffic "balance" requirement for settlement-fee peering? Cloudflare sends so much data out that they probably want to get more sent to them so things balance out. The more garbage traffic, the better! :) I'm totally out of my bailiwick here, it can't be that simple, can it?
- joeseeder 9y agoRIPE Labs ran some tests back in a day, on this subnet, with interrsting conclusions https://labs.ripe.net/Members/franz/content-pollution-18 https://labs.ripe.net/Members/franz/content-pollution-18
- taurath 9y agoSuper interesting! It’s the IP address space equivalent of a no mans land - a dead zone where stray packet radiation will cook your servers. One can only wonder what goes on under the surface..
- robertcope 9y agoI didn't have any problem getting off the ATT Fiber network, but it was getting dropped down the line towards the datacenter.
- notafxn 9y agoWhen 1.1.1.1 launched here in Spain, it was inaccessible from several major carriers. Some had network-wide routing problems to that IP address, and some had installed CPEs that included static routes to 1.1.1.0/24 and stuff like that, probably for internal purposes. Nothing of this strikes me as odd, let alone malicious. It's such a weird IP range, I even remember having my LAN configured as 1.0.0.0/24 at some point, because who would ever use those IP addresses? Also reminds me of when Spanish ISPs were given IP ranges by RIPE for their customers beginning with 37.* -- those had never been used, so many network administrators had them added to their bogon list, which meant for those customers lots of web pages were inaccessible. The solution was to reboot their CPEs until they got a good ol' IP address from the ol' ranges :D
- darklajid 9y agoNothing like that is an excuse. You have 10.0.0.0 for that - it's huge and you can use it for whatever you want without stepping on anyone's toes. There are absolutely zero reasons I've seen so far (I'd be interested to hear abstruse ones? "I thought who cares" isn't one) to avoid using one of the private ranges. 10.0.0.0/8 isn't even noticeably harder to remember or type, really.
- tssva 9y agoUsing the 10/8 or any of the other RFC1918 had great potential to step on their customers toes. That is exactly why rightly or wrongly they used the 1.1.1.0/24 range. Hardware manufacturers generally used the range for interfaces that were local to the device and often only used on interfaces internal to the device. They knew this equipment would be deployed into environments where RFC1918 addressing would be used but they had no idea what RFC1918 address ranges, so using addressing from the RFC1918 networks meant potentially impacting their customer's data. They chose to instead use addressing which at the time they believed would not impact their customers. APnic is not blameless here. They knew the issues with this space when it was assigned to them as a research network. For quite awhile they allowed Google to advertise the space and collect data on it's usage. I assume Google no longer was providing the infrastructure to do so and APnic saw an opportunity to have someone collect data for them for free. Collecting data on traffic sent to this ip range is one thing but approving its use for a service available to the public knowing the accompanying issues much of the public would have accesssing it is in my opinion not responsible use of a research network.
- gerardnll 9y agoFew ISPs (Movistar, Vodafone) here in Spain also use 1.1.1.1 for router internal stuff... So we can only use 1.0.0.1 at the moment.
- DominoTree 9y agoSemi-related: once some hardware arrives, I should be able to finish finding a way to rip the 802.1x certs off of the 5286AC so we can use our own routers, or at least put these things into a proper bridged mode. https://spun.io/2018/03/18/getting-into-the-pace-5268ac-router-part-1/ https://spun.io/2018/03/18/getting-into-the-pace-5268ac-rout...
- jmuguy 9y agoI'm sure you're pretty far down the rabbit hole of dealing with the 5268AC but the "DMZ Plus" mode does seem to bridge. I have a Ubiquiti Edgerouter on the other side with OpenVPN setup, public IP, etc. All I needed to do was enable DMZ Plus and turn off wifi.
- phil21 9y agoDMZ Plus doesn't bridge - it just fakes it by essentially 1:1 NAT'ing a public IP to your router. You are still using it's internal routing and NAT tables, which to be charitable are problematic. You also lose the ability to do ipv6 - not that AT&T's ipv6 implementation is worth even using at this point. I just moved from Comcast to AT&T gige and the speeds are certainly great - but this CPE (and horrible v6 even if it worked) is seriously making me reconsider.
- jmuguy 9y agoAh, that is a bummer. I certainly miss the days of just buying my own dumb cable modem and never thinking about it again.
- ryan-c 9y agoYou can bypass these routers with an 801.1x MitM attack. You basically put it behind your Linux router and bridge only the EAPoL frames, then do DHCP from your own router. I posted some details about this on DSLReports[1]. I have been using this method with AT&T GigaPower fiber in Austin, TX for two years and it's been totally stable and free of problems. That said, I'd love to be able to extract the cert and not have to do this. 1. https://www.dslreports.com/forum/r30708210-AT-T-Residential-Gateway-Bypass-True-bridge-mode https://www.dslreports.com/forum/r30708210-AT-T-Residential-...
- swiley 9y agoNever ever ever use the hardware firewall that your ISP tries to sell/rent to you. Better still is to use an old Linux desktop with IPtables doing masquarade, that's what these cheap things are doing anyway but now you have control.
- Klathmon 9y agoSadly you can't with AT&T, they have a cert on their routers and won't give out any IPs to anything else other than their hardware.
- eli 9y agoT-Mobile US did (and maybe still does?) hijack 4.2.2.2 and 8.8.8.8 to silently redirect them to their own crappy DNS server. http://esd.io/blog/t-mobile-dns-hijack.html http://esd.io/blog/t-mobile-dns-hijack.html
- Coincoin 9y agoAre people really that surprised 1.1.1.1 won't work everywhere? I suck at networking, but I worked that job just long enough to know there is some pretty horrible stuff such as 1.1.1.1 being hard coded all over the place, even in high-end hardware. AT&T's crime isn't really that they are stealing stuff, but rather that they aren't doing anything to fix the millions of legacy stuff with broken 1.1.1.1 Granted, the most plausible explanation is they thought: "To hell with it, it's broken, we are going to use it."