12 ms·
Ask HN: Is Comcast having internet problems?
I believe others are having problems too: https://news.ycombinator.com/item?id=15637299
Update: The issue seems to be around 50% packet loss.
I called Business Support and the person was unaware of the issue, so please also call in to report the issue so Comcast starts to realize how much of a problem this is.
I had the support person ping facebook.com from my modem which showed packet loss. I asked them to send me to the next tier of support which they did. (This ends up being a ticket where someone will call me back.)
The issue started for me around 12:50pm Eastern time.
Update 2:10pm eastern: There's another thread on the homepage: https://news.ycombinator.com/item?id=15637607 (update: 2:26pm the other thread was removed from the home page)
Update: I contacted Comcast Network Operations Center. They confirmed they're aware of the issue and working on fixing it, but didn't provide any more details.
Update 2:30pm eastern: The issue appears to be resolved.
- senthilrajndren 9y agoYes it is having issues across USA CHeck this https://www.downstatus.com/comcast-xfinity-outages https://www.downstatus.com/comcast-xfinity-outages
- noncoml 9y agoComcast has been so unreliable for me the last year, that I pay for a second backup DSL connection from AT&T.
- ozfive 9y agoThis was caused by a misconfiguration at L3 NETWORKS which btw i should add is now owned by centurylink. Go figure.
- exikyut 9y agoWow. I'm reading through what's left of this thread and quite amazed at the impact. There should be a service sysadmins can run that sends controlled pings, download/upload requests, etc, to specific hosts. The idea being that the results are shared publicly. -- Copypasting https://www.wired.com/story/how-a-tiny-error-shut-off-the-internet-for-parts-of-the-us/ https://www.wired.com/story/how-a-tiny-error-shut-off-the-in... (dated today): -------------- A YEAR AGO, a DDoS attack caused internet outages around the US by targeting the internet-infrastructure company Dyn, which provides Domain Name System services to look up web servers. Monday saw a nationwide series of outages as well, but with a more pedestrian cause: a misconfiguration at Level 3, an internet backbone company—and enterprise ISP—that underpins other big networks. Network analysts say that the misconfiguration was a routing issue that created a ripple effect, causing problems for companies like Comcast, Spectrum, Verizon, Cox, and RCN across the country. Level 3, whose acquisition by CenturyLink closed recently, said in a statement to WIRED that it resolved the issue in about 90 minutes. "Our network experienced a service disruption affecting some customers with IP-based services," the company said. "The disruption was caused by a configuration error." Comcast users started reporting internet outages around the time of the Level 3 outages on Monday, but the company said that it was monitoring "an external network issue" and not a problem with its own infrastructure. RCN confirmed that it had some network problems on Monday because of Level 3. The company said it had restored RCN service by rerouting traffic to a different backbone. The misconfiguration was a "route leak," according to Roland Dobbins, a principal engineer at the DDoS and network-security firm Arbor Networks, which monitors global internet operations. ISPs use "Autonomous Systems," also known as ASes, to keep track of what IP addresses are on which networks, and route packets of data between them. They use the Border Gateway Protocol (BGP) to establish and communicate routes. For example, packets can route between networks A and B, but network A can also route packets to network C through network B, and so on. This is how internet service providers interoperate to let you browse the whole internet, not just the IP addresses on their own networks. In a "route leak," an AS, or multiple ASes, issue incorrect information about the IP addresses on their network, which causes inefficient routing and failures for both the originating ISP and other ISPs trying to route traffic through. Think of it like a series of street signs that help keep traffic flowing in the right directions. If some of them are mislabeled or point the wrong way, assorted chaos can ensue. Route leaks can be malicious, sometimes called "route hijacks" or "BGP hijacks," but Monday's incident seems to have been caused by a simple mistake that ballooned to have national impact. Large outages caused by accidental route leaks have cropped up before. "Folks are looking to tweak routing policies, and make mistakes," Arbor Networks' Dobbins says. The problem could have come as CenturyLink works to integrate the Level 3 network, or could have stemmed from typical traffic engineering and efficiency work. Internet outages of all sizes caused by route leaks have occurred occasionally, but consistently, for decades. ISPs attempt to minimize them using "route filters" that check the IP routes their peers and customers intend to use to send and receive packets, and attempt to catch any problematic plans. But these filters are difficult to maintain on the scale of the modern internet, and can have their own mistakes. Monday's outages reinforce how precarious connectivity really is, and how certain aspects of the internet's architecture—offering flexibility and ease-of-use—can introduce instability into what has become a vital service.
- amorphid 9y agoMildly off topic... For Chromecast users, I discovered that there's a wired Ethernet dongle you can use to eliminate the need for wifi... https://store.google.com/us/product/ethernet_adapter_for_chromecast https://store.google.com/us/product/ethernet_adapter_for_chr... This past weekend my wifi was super slow, to the point streaming music would have buffering issues on my Chromecast. A quick speedtest.net assessment showed wifi was only getting 2mbs (2 meters from the router) and wired was getting 50mbs. I googled for options, and found the dongle.
- angrydev 9y agoComcast user in SE Michigan here, can't reach certain websites (reddit, cnn, basecamp)
- rbritton 9y agoSame here in Washington.
- ghshephard 9y agoComcast Business user in Ann Arbor, MI - some sites are fast (Ycombinator) but a lot of others (Atlassian) are dog slow. A 40 Megabyte download from https://dbeaver.jkiss.org/download/ https://dbeaver.jkiss.org/download/ that normally takes 30 seconds is on target for 2 hours.
- cglace 9y agoSame in Atlanta.
- Shelnutt2 9y agoSame in Boston
- mikesickler 9y agoSame in Philly
- cls59 9y agoSame in San Jose.
- deleted 9y ago[deleted]
- uncled1023 9y agoI'm in Northern California and also having issues. One thing I noticed is that it is only Downloads having issues. My uploads are as fast as ever. I then connected to my VPN, and everything is working great. I think it is their routing/DNS that is having the issues, not the connection itself.
- methodover 9y agoSame in San Francisco
- justinlink 9y agoLot of packet loss. Pennsylvania Comcast Business Class. Twitter has a lot of people complaining all over the country when searching the tag comcast.
- jetrink 9y agoRCN is also having issues[1]. I'm seeing 50%+ packet loss. 1. https://twitter.com/RCNconnects/with_replies https://twitter.com/RCNconnects/with_replies
- oregontechninja 9y agoSame in Albany, OR. Any theories or answers as to why we're seeing such a performance loss?
- Todd 9y agoComcast Business is bad near Seattle
- aacook 9y agoSame issues in Plymouth, MA. Noticed it first w/ Stripe but now other sites like Reddit too.
- kevin2r 9y agoSouth Florida as well, can't reach some websites.
- dfcab 9y agoSame here, been crawling since around 1pm EST
- stormcode 9y agoBoston, Comcast Biz. Same issues.
- brianwawok 9y agoSouth Bend, IN is no bueno here.. both from comcast business and comcast residential
- Pieman103021 9y agoI currently can't reach doodle, something is definitely up.
- booleanbetrayal 9y agoBoulder, CO = issues
- dhess 9y agoYes. Comcast Business, SF.
- funwithjustin 9y agoOakland checking in. Can't reach about half of the internet, dropping a lot of packets.
- toomim 9y agoBerkeley too.
- funwithjustin 9y agoGuess we should just grab a pint and wait for this whole thing to blow over.
- charisma123 9y agoEmeryville too.
- gregimba 9y agosome sites are timing out for me in WA
- cjbenedikt 9y agosame in Baltimore
- 8000800008 9y agoSecaucus, Edison NJ also having issues
- aaronmiler 9y agoComcast home customer in Portland OR Netflix servers responding and streaming fine, but almost everything else is slow
- funwithjustin 9y agoSerious question - what could cause this? I don't know much about network configuration or DNS. Could it be a coordinated attack on Comcast's infrastructure somewhere? Seems a little crazy that they'd be experiencing problem nation-wide with no redundancy baked into their system. I tried manually flipping over to Google's DNS servers with the same result.
- _rpd 9y ago> what could cause this? It could be ... The Backhoe, The Internet's Natural Enemy (2006) https://it.slashdot.org/story/06/01/19/1643215/the-backhoe-the-internets-natural-enemy https://it.slashdot.org/story/06/01/19/1643215/the-backhoe-t...
- Shelnutt2 9y agoSince we are resolving dns, we know its not a dns issue. This appears to be pack loss / routing issue. The most likely cause of this is someone has leaked internal routes, or leaked bgp. I don't see anything from bgpmon (https://twitter.com/bgpmon https://twitter.com/bgpmon) or bgpstream (https://twitter.com/bgpstream https://twitter.com/bgpstream) for North America though. There have been multiple bgp leaks today but nothing that I can see that effects comcast/rnc. It does appear at least some site I'm seeing are being routed through Europe. traceroute kayak.com traceroute to kayak.com (185.6.169.69), 64 hops max, 52 byte packets 2 96.120.68.17 (96.120.68.17) 7.840 ms 8.995 ms 8.354 ms 3 96.108.154.229 (96.108.154.229) 8.136 ms 9.174 ms 8.325 ms 4 be-23-sur03.westroxbury.ma.boston.comcast.net (69.139.221.226) 9.879 ms 7.478 ms 7.667 ms 5 be-20-ar01.needham.ma.boston.comcast.net (68.85.106.21) 9.527 ms 12.000 ms 8.524 ms 6 * * * 7 ae-1-16.bar1.zurich1.level3.net (4.69.142.129) 118.578 ms 118.947 ms 118.229 ms 8 ae-1-16.bar1.zurich1.level3.net (4.69.142.129) 127.931 ms 116.424 ms 117.852 ms 9 nexellent-a.bar1.zurich1.level3.net (213.242.82.110) 118.340 ms 117.555 ms 115.950 ms
- Eyght 9y agoSeems like the UK and Germany are under relatively heavy load right now. https://www.akamai.com/uk/en/solutions/intelligent-platform/visualizing-akamai/real-time-web-monitor.jsp https://www.akamai.com/uk/en/solutions/intelligent-platform/...
- tclancy 9y agoDover, NH Comcast, same here.
- albeebe1 9y agoMerrimack NH same
- donutdan4114 9y agoManchester, NH... Sllooowowww....... Droppin' packets like it's hot.
- trebor 9y agoSame trouble in Frederick, MD. Packet loss appears to be high too.
- caseyohara 9y agoComcast residential customer in Denver, CO. Lots of sites intermittently slow/unreachable right now.
- kreisys 9y agoBell Canada business customer in Ottawa. Getting about 50% packet loss to about 50% of the internet.
- deadcast 9y agosame in Modesto, CA
- chipuni 9y agoSame problem in Hayward (i.e. Bay Area), CA, USA.
- jipiboily 9y agoGetting a very similar issue with Bell Canada. So similar I'm wondering if they are Comcast customers themselves.
- wiremine 9y agoWest Michigan is seeing similar: 40% to 50% packet loss for most websites.
- raincom 9y agoSame in San Jose, CA, Berryessa area: can't access reddit; however speedtest shows 210M upload, 10M download. I use Google DNS for my apartment network.
- dashwin 9y agoIt may be packet loss on certain routes. I resorted to using VPN and that resolved the problem at-least while over VPN. Without VPN its a CRAWL for many sites.
- bguillet 9y agoMillbrae, CA, same problem with Comcast
- funwithjustin 9y agoAs of 11:30 PST in Oakland on Xfinity, things appear to be coming back to normal. Anybody else?
- collinmanderson 9y agoYes, things appear to be back now.
- Twisol 9y agoSame, shortly before you in Sacramento on (I believe) Comcast.
- maccam94 9y agoI was also seeing problems with Comcast in San Francisco, but they resolved just before I found this thread (~11:30 PST)