22 ms·
Possible BGP hijack of 1.1.1.1
- highace 8y agoWhat does this mean for those unfamiliar?
- jpollock 8y agoIf true it means that Cloudflare's DNS server can't be trusted.
- ancarda 8y agoDo you mean it can't ever be trusted? Or just right now? BGP hijacking isn't that difficult to pull off. I hope you're not trusting 8.8.8.8 either: https://twitter.com/bgpmon/status/445266642616868864 https://twitter.com/bgpmon/status/445266642616868864
- Latteland 8y agoThat's not correct, anyone could 'accidentally' do this to every other provider. It's not a special weakness of cloudflare.
- jpollock 8y agoI'm sorry, I went for brevity because someone was asking for possible impacts. If I were using 1.1.1.1 as a DNS server and saw this news story, I would change to a different DNS server until the problem was resolved. My goal was to provide actionable information quickly. I never said it was specific to Cloudflare. :) It is specifically a mistake which would break an assumption - that putting 1.1.1.1 into your resolver results in an answer from Cloudflare. DNS doesn't necessarily have any protections (not current, so maybe they were added?), so the only level of protection is that the IP address routes UDP traffic where we expect it to. It also isn't a long-term problem, it only remains for the length of time the route is wrong. It could also be argued that we're already trusting every router between the device and 1.1.1.1 anyways, so there's not much difference. Except that there's already a trust relationship between those groups, and the new route subverts them. It's the same level of risk if someone had done a BGP hijack of any backbone router.
- bonyt 8y agoTraffic meant to go to 1.1.1.1 (cloudflare DNS) could be routed elsewhere. Since this is a common DNS server, this could be used to alter domain resolution for people that use it.
- HankB99 8y agoI'm assuming this would/could be done by a malicious party in order to substitute different IP addresses for some sites in an attempt to direct traffic for nefarious purposes. If my host is configured to use DNSSEC would that prevent sites from resolving? If DNSSEC is not employed and a connection is directed to a malicious site (using https) wouldn't that prevent the connection? (I'm afraid I'm out of my depth on the implications of this aspect of networking and wondering about the security implications for me since I'm using Cloudflare DNS servers.)
- Operyl 8y agoFor DNSSEC: it depends, I don’t think many clients hardfail yet. For HTTPS: if you can BGP attack, theoretically you could get a TLS certificate issued. There’s a lot of ifs on both those roads, though.
- snuxoll 8y agoProbably a good use case to pin the certificates for your upstream DNS resolvers if you're using DNS over TLS/HTTPS.
- Operyl 8y agoYeah. As of right now there’s no absolutely fool proof way to ensure all clients aren’t possibly going to slip through some crack, and I’m not sure if we’ll ever get there.
- deleted 8y ago[deleted]
- dec0dedab0de 8y agocould be a problem for wifi captive portals that redirect to 1.1.1.1, but most of them would never route to begin with. However, I believe the real issue is that it is a free DNS server, so someone could redirect all domains that do not use ssl pinning.
- czbond 8y ago1.1.1.1 is a DNS resolver that does not track activity. A BGP compromise means that someone could have compromised it and redirect/intercept traffic of those trusting it to be Cloudflare.
- dharmab 8y agoA BGP attack does not compromise the destination host. It reroutes (some) traffic destined for the host. Any traffic using TLS to establish destination authenticity (e.g DNS TLS, DNS over HTTP) or content authenticity (e.g. DNSSEC) would detect the attack, while other types of traffic (traditional DNS) could be exploited.
- uremog 8y agoIsn't the point that the attacker could compromise that promise of non-tracking? They could track whatever is routed through them and then forward them on to the legitimate destination.
- vbernat 8y agoThis could be a first step to compromise TLS traffic as well: https://www.princeton.edu/~pmittal/publications/bgp-tls-hotpets17 https://www.princeton.edu/~pmittal/publications/bgp-tls-hotp...
- tptacek 8y agoDNSSEC does not in fact mitigate BGP attacks, because to the extent it works at all, DNSSEC protects only the mapping between IP addresses and names. A BGP attacker controls the semantics of the addresses themselves, and can simply leave DNS pointing where it's supposed to, but hijack the underlying address. TLS, on the other hand, does address this attack, because controlling all the traffic to a TLS-protected site still doesn't give you a private key that produces a valid signature on a certificate for that site.
- Lennie 8y ago"Any traffic using TLS to establish destination authenticity (e.g DNS TLS, DNS over HTTP)" Well, unless you can also fool Let's Encrypt from all their locations around the world. Then you can get a Let's Encrypt certificate.
- sudhirj 8y agoSomebody other than Cloudflare (the current holders) of the IP is receiving some of the traffic meant for it. Usuall hijack reasons are to point users to fake sites (I point yourbank.com to my own server) and phish.
- koolba 8y ago1.1.1.1 is the primary IP for CloudFlare’s new DNS service. A BGP Hijack is when the destination route for that IP changes from it’s legit target to somewhere else. Often times it’s accidental but could be part of a wider attack. In the case of DNS it’s particularly nasty as the attacker would control address resolution (say redirecting traffic for your bank to a phishing site) for everyone using 1.1.1.1 without more specific mitigations. Combined with long DNS cache times this could be a problem for a while.
- polpo 8y agoCloudflare operates a public DNS server on 1.1.1.1 that has gotten a lot of attention since it was launched a few months ago. If a bad actor hijacks it, they can answer DNS queries with malicious answers, similar to the Amazon Route53 hijack that was used to redirect an Etherium wallet site to a fake server: https://www.internetsociety.org/blog/2018/04/amazons-route-53-bgp-hijack/ https://www.internetsociety.org/blog/2018/04/amazons-route-5...
- ddtaylor 8y agoThey could also simply log all the information and have made the server appeared to continue to operate as normal, since most fooling with the DNS packets would yield certificate errors for many sites (Google, YouTube, etc.) Most of the time someone comes out and says the BGP hijacking was an accident. A bit of a Hanlon's razor situation.
- webtodded 8y agopersonally im not super familiar but i found this recent article from cloudfare that gives some background information https://blog.cloudflare.com/bgp-leaks-and-crypto-currencies/ https://blog.cloudflare.com/bgp-leaks-and-crypto-currencies/
- wmoses 8y agoIn other words, some entity that is not cloudflare claimed to have the best route to (some of?) cloudflare's IP ranges, including 1.1.1.1. If malicious, this could be someone trying to redirect 1.1.1.1 traffic elsewhere. There have also been a lot of historical examples of misconfiguring BGP (the way big internet networks talk to each other and discuss where to send packets), such as a florida ISP accidentally claiming the best route to some major internet service and getting flooded with everyone's traffic until it died. BGP is also really insecure (back in 2008 pakistan effectively brought down youtube for instance through BGP -- which doesnt require any authentication to claim you have the best route to X).
- floatingatoll 8y agoIt’s like someone posting a “Turn left for Highway 123” road sign right next to a legitimate “Turn right for Highway 123” sign. Traffic snarls result, since most drivers don’t check the DOT security sticker on the back of both signs and find it missing from the fake Left one.
- walrus01 8y agoSignificantly simplified: BGP4, which is one of the fundamental building blocks of the global Internet, relies on trust between BGP peers. ISP A says to ISP B, their peer, "hey I'm responsible for this chunk of publicly routable IP space, please send all traffic to ASN number N for this particular block". This works as long as everyone configures their IP space announcements and prefix-list filters correctly. A lot of less clueful ISPs in the world do not verify the IP space announced to them by their peers (BCP38 is your friend!). This results in things like the time that a telecom in Pakistan hijacked the IP space for most of Youtube about ten years ago and successfully DDoSed themselves, while also causing a major youtube outage. https://www.google.com/search?q=pakistan+bgp+hijack+youtube&ie=utf-8&oe=utf-8&client=firefox-b-1 https://www.google.com/search?q=pakistan+bgp+hijack+youtube&... This will keep happening until various ISP peers properly implement prefix-list filtering, ACLs on their edge BGP connections, and verifying peer announcements via things like various route registries.
- jacquesm 8y agoAh! That may have been the reason why my site wasn't resolving earlier today. It was the weirdest situation with people from all over the planet complaining without any apparent pattern, a RIPE check of the site from 10 different locations showed no issues in connectivity. Thanks for posting this.
- minxomat 8y agoNo, the issue persists. While I can access your site from mobile and residential connection, any static business connection fails. No 1.1.1.1 involved. I tested this with two different Fibre connections (Berlin).
- jacquesm 8y agoIs the BGP hijack over?
- ColinWright 8y agoAs a data point - your site still isn't loading for me. IP : 79.69.113.214 Time: Tue May 29 15:28:30 BST 2018
- jacquesm 8y agoI wonder if it is a resolution issue or an access issue. What happens when you go to: http://62.129.133.242/ http://62.129.133.242/ ? That should come up with a 'domain for sale' page, that's the same server.
- carbocation 8y agoNot loading for me from MA, USA. $ httpstat http://62.129.133.242/ 2018/05/29 10:54:04 unable to connect to host 62.129.133.242:80: dial tcp 62.129.133.242:80: connect: connection timed out
- 8y ago
- ancarda 8y agoHow effective is this? Looking at https://bgp.he.net/ip/1.1.1.1 https://bgp.he.net/ip/1.1.1.1, 1.1.1.0/24 is apparently "ROA Signed and Valid". I don't know a lot about BGP. Does this mean hijacking this subnet is a bit harder than unsigned ones because some or all ISPs verify this announcement? Or is it faster/easier to detect? Maybe a wider question: is there some way to prevent BGP hijacking?
- walrus01 8y agoBasically, the bigger Chinese ISPs that are upstream of this small one which is making the false 1.1.1.0/24 announcement are not actually verifying that this small ISP is allowed to announce the space. As for prevention, the only thing that will work is proper use of IRR/route registries and RPKI validation of peer announcements. Which a great many ISPs do not currently do. https://www.noction.com/blog/bgp-hijacking https://www.noction.com/blog/bgp-hijacking The other method is more blunt, and can be more effective if the people with 'enable' on various ASNs' core and edge routers actually have a spine. ISPs which repeatedly announce space that they're not allocated (as per RIPE, ARIN, APNIC, AFRINIC records) should be depeered by their local peers, and their owners/operators publicly shamed. It's a reputation thing. As a neighbor of other, more clueful ISPs, it's basically the same thing as being a bad neighbor by leaving garbage all over your front lawn and causing a public nuisance with loud parties and trashy behavior.
- xmodem 8y agoIn short, not many networks are checking signatures because not many networks are publishing them. Take a look at this presentation from 2009. http://www.ausnog.net/sites/default/files/ausnog-03/presentations/ausnog03-michaelson-rescert.pdf http://www.ausnog.net/sites/default/files/ausnog-03/presenta... I've been out of the space for almost as long, so would love to be wrong, but I think its fair to say that not much has improved on this front since then.
- hvoiiita 8y agoRADb or some other RIR database registration is what my company requires. This wont really stop bad actors, however.
- zimbatm 8y agoASN 58879 belongs to Shanghai Anchang Network Security Technology Co.,Ltd (China) according to https://ipinfo.io/AS58879 https://ipinfo.io/AS58879 website: https://www.anchnet.com/ https://www.anchnet.com/
- fiber 8y agoI doubt that this is a genuine hijacking attempt. All it takes is a Cisco router and some IT admin making up an address.
- floatingatoll 8y agoHas the hijacker previously hijacked other prefixes in the past, or is this a one-time event for them?
- bodyfour 8y agoAgreed. As many pointed out when the 1.1.1.1 DNS service was introduced, it's an address that is often used (incorrectly) as an internal or temporary IP. Then all it takes is a slight mistake in your route redistribution and suddenly you can find yourself accidentally announcing the prefix to eBGP. I wouldn't be surprised if this becomes a semi-regular occurrence.
- walrus01 8y agoHanlon's razor applies to a great degree. I have no doubt that there are a great many enterprise-type organizations that have been using 1.0.0.0/8 internally and cluelessly for a very long time.
- n1c 8y agoInteresting! My ping to that address went terrible for a brief window today - https://i.imgur.com/KjCcBeT.png https://i.imgur.com/KjCcBeT.png Wonder if this was the cause. *edit: I'm in Cape Town and the ping looks what was routing to a DC down the road decided to go to Europe instead.
- ancarda 8y agoHey, what software are you using to collect the data and display that graph? I wouldn't mind running something on my server that could notify or log if 1.1.1.1 (and other services I rely on) are slow or down.
- amaccuish 8y agoAnd that is why I'm using dns over tls :)
- moviuro 8y agoNot enough. You have to check the certificate's fingerprint, along with its validity. TLS is not a silver bullet. If an attacker controls the host behind what everyone believes to be 1.1.1.1, nothing is to prevent them from applying for a legit certificate.
- tialaramex 8y agoNothing prevents them from applying but... * They need to do that, and get the resulting certificate, and install it, during the attack. The weirder the product (and certificates for IP addresses are relatively weird) the more humans end up involved in your order, and humans are slow. * This leaves a smoking gun in the Certificate Transparency logs. So we all get to know (in maximum 24 hours but usually the reality will be minutes) about this extra certificate.
- peterwwillis 8y ago1) would take about 20 seconds, thanks to Let's Encrypt, but probably only marginally more time for some other CA with an API. 2) Who exactly is staring at CT logs and going "oh, I don't remember this domain using this CA, maybe I should investigate this" ? Sure there's a record of it. Doesn't really matter during an attack, because public attacks like this aren't intended to last long. All you need is a half hour or less to steal a couple hundred million from a bank, or cryptocurrency wallet, using this attack. That's more than enough incentive for most unscrupulous 3rd world hackers. If you're an authoritarian government, you could require CAs in your country to selectively quiet CT logs by certain users, and just issue certs willy-nilly for your private government org for MITM purposes. Google Chrome would detect them for Google-owned properties, but smaller sites would never know. And spy agencies can use this at their leisure and basically never be held accountable, because world politics. Let's face it. The CA system is a joke and BGP is the butt of it.
- amaccuish 8y agoDoes anyone else find it sort of beautiful watching replays of events like this? It's amazing to watch how the routers organise themselves, making and breaking connections when needed.
- kchr 8y agoThere are at least two of us, dear friend!
- davidkuhta 8y agoThree's company.
- digi_owl 8y agoI must admit, what drew me to computing back in the day was the low level networking stuff.
- chillingeffect 8y agoyes, what JS library does that graph drawing and animation? Or a similar one?
- svnsets 8y agoLooks like they are using https://bgplayjs.com/ https://bgplayjs.com/ for that graph.
- Florin_Andrei 8y agoI mean, this is what BGP does.
- cortesoft 8y agoRight, and it is pretty cool to watch is the point.
- floatingatoll 8y agoWould this affect certificate-validating clients doing DNS-over-HTTPS to 1.1.1.1 — doesn’t it have an ipAddress certificate and demand HTTPS resolution only?
- moviuro 8y agoWell, if you control the host behind the IP, you could have any CA issue a challenge, and successfully pass it (e.g. if Let's encrypt uses the erroneous routes). So no. The only thing protecting you would be to have the expected hash of the certificate you expect to see (TOFU - Trust on First use, though you're screwed if you didn't contact 1.1.1.1 before the incident!).
- floatingatoll 8y agoIs there a CAA equivalent for ARIN assignments?
- Operyl 8y agoI assume you’re looking for: https://tools.ietf.org/html/bcp38 https://tools.ietf.org/html/bcp38
- floatingatoll 8y agoNope, that’s not what I’m looking for at all.
- devicenull 8y agoRPKI, but it's barely used
- floatingatoll 8y agoNope, that does not cover certificate issuance for IP addresses in that range.
- 8y ago
- deleted 8y ago[deleted]
- jacksmith21006 8y agoCurious how is this different than the similar? issue with Amazon route 53 getting hijacked not too long ago?
- akw28888 8y agoI'm using AnchNet's services. And We've asked AnchNet when I recieved a e-mail from our BGPMon. They said their staff was configured a wrong config on router. Also they don't know 1.1.1.0/24 is used by CloudFlare&APNIC. So they used this prefix to test.
- tazjin 8y agoIt shouldn't really matter who is currently using that net, it's not a private range :/
- jedisct1 8y agoTell Cisco.
- walrus01 8y agoBoth Cisco and Juniper use the standardized documentation IP ranges in their example/lab configurations and training materials. https://tools.ietf.org/html/rfc5735 https://tools.ietf.org/html/rfc5735 https://tools.ietf.org/html/rfc5737 https://tools.ietf.org/html/rfc5737
- detaro 8y agoCisco did recommend to use 1.1.1.1 (or did default to? commentary online is unclear), but in a different space: for the DHCP server with DHCP proxying in their WLAN products. They might be referring to that.
- alberts00 8y agoI have been going through Cisco Netacad materials for last 2 years and I've seen 1.1.1.1 in examples. Though I just did a search and did not see 1.1.1.1 mentioned anymore.
- sdfgdfhjdgj 8y agoWhy let people access BGP that don't even know that 1.0.0.0/8 or 1.1.1.0/24 are part of the public internet or that decide they can use random prefixes to "test" things? :-/
- deleted 8y ago[deleted]
- walrus01 8y agoNetwork engineer here: I'm going to guess that this is a mistaken effort on the part of a Chinese ISP or the GFW to hijack traffic to 1.1.1.1 internally within China, but probably not intended to propagate beyond the major Chinese international-transit-ISP's connections to the global Internet. BCP38 is your friend.
- unethical_ban 8y agoIt's been a while so I should re-read it, but I thought BCP38 was best applied at the terminal ISP/client network level. If you're a transit network, you can't do that kind of fitering because you have a legitimate chance of forwarding traffic to and from any network address.
- walrus01 8y agoMore about the principles of not just ingress filtering, but the same as BCP38, but as an ISP with its own ASN, applied to your own egress: a) don't announce shit you don't own b) know how to set up ACLs and prefix-list filters on your own egress IP space announcements which face towards your peers and IP transit upstreams. Conversely, as a big ISP which has many small ASNs downstream of it, be responsible and set up filters on your own ingress which prevent your customers from announcing mistaken shit to you. Using an example of a clueful and attentive major ISP: For example if you are a small to medium sized regional ISP and buy IP transit from NTT, one of the world's top-ten global commercial transit providers, they actually do take the time to verify each and every prefix you announce to them and will require an interaction with their NOC if you want to announce a new /22.
- akw28888 8y agoAS58879 is used for AnchNet's international services, like Hong Kong, Los Angeles. Not used in China mainland. They used AS55994 in China mainland. In fact, China's ISP do filter via prefix and they all enable URPF. In China, IDC can't announce non-cnnic addresses
- throw9991999 8y agoI use 1.1.1.1 Do I need to do anything? Can I just continue using it or do I need to clear some cache etc?
- spacenick88 8y agoThat awkward moment when you read an IP and the first thought is "But that belongs to Cloudflare I read about this"
- ChuckMcM 8y agoSo who is going to tell the 13 peers that they should not accept BGP path advertisements for 1.1.1.0 from anyone but Cloudflare?
- solotronics 8y agoLarge companies misuse "unassigned" space all the time. I have heard engineers at my work propose using the non public routed DOD /8 before. Not on my watch!
- vivalibre 8y agoAnd I can confirm that this is done at some Very Large ISPs. Seen it with my own eyes, e.g. 30/8
- TheAceOfHearts 8y agoIf you happen to know why, could you explain their reasoning? In what ways are 10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16 insufficient?
- solotronics 8y agoI help design one of big 3 cloud providers and we're about to run out of private space for customer IPv4. We are addressing this is in a number of ways but I think others have run into this same issue.
- vesche 8y agoAre people here really using 1.1.1.1 as a DNS server...? Do people here _really_ think that Cloudflare isn't giving your data away to _someone_? I have been using DNS servers from OpenNIC for sometime now, and I will continue to.