12 ms·

Network Outage in EU affecting AMS1 and AMS2

16 points by ibarrajo 12y ago

From Digital Ocean: At this time we're experience a network outage in our AMS1 and AMS2 datacenters. With secondary impact affecting customers in Europe. As a result, you may experience latency, connectivity issues, or slow pings. Preliminary investigation indicates that Telias Transatlantic cable are down. We are working to resolve the issue and apologize for any interruption this causes for you. Mon May 19 20:29:32 2014

nodesocket 12y ago
As a side effect, anybody else getting intermittent false alarms from Pingdom?
- peterwaller 12y ago
  You're not the only one.
  - thehodge 12y ago
    I had this issue on the new beeper system, I switched back to the old system and have had no issues since.
- bobfunk 12y ago
  yeah, got a bunch - guess they're not strictly speaking false, it's just not something most of us can do anything about...
diorray 12y ago
It's not Digital Ocean's fault. It seems like CloudFlare, Twitter and Facebook's EU servers are down too..
- sampo 12y ago
  I am in Europe, and I can reach Facebook's EU servers just all right :-).
  - deleted 12y ago
    [deleted]
  - deleted 12y ago
    [deleted]
carlio 12y ago
A bit of tracerouting and some twitter searching[0] suggests that the issue likes with Telia.net. There are some unconfirmed suggestions that a cable was severed[1], which I'm not sure I believe yet but certainly any packet going via telia.net is having a bad time. Edit: Even CloudFlare are reporting the cable cut[2] so... gosh. [0] https://twitter.com/search?q=telia&src=typd https://twitter.com/search?q=telia&src=typd [1] https://twitter.com/CurseNetworkSys/status/468493477542494208 https://twitter.com/CurseNetworkSys/status/46849347754249420... [2] https://twitter.com/CloudFlare/status/468493714134822912 https://twitter.com/CloudFlare/status/468493714134822912
abritishguy 12y ago
Transatlantic cable has been cut, lots of sites affected.
nodesocket 12y ago
Akamai has a nice visualization map. Look at all that red in Europe. http://www.akamai.com/html/technology/dataviz1.html http://www.akamai.com/html/technology/dataviz1.html
deleted 12y ago
[deleted]
ccozan 12y ago
In Germany we felt it for about 15mins, now the routing is adjusted and we are back to normal. This is a nice reminder that the Internet was designed to resist to such disturbances.
lucaspiller 12y ago
Related discussion: https://news.ycombinator.com/item?id=7769587 https://news.ycombinator.com/item?id=7769587 From the UK everything seems to be working normal now, including access to my instance in AMS1.
ultrasaurus 12y ago
At PagerDuty we saw our alerting volume double between 19:50 and 20:00 UTC (about 90 minutes ago).
raiyu 12y ago
It looks like from our end that there was a large trans-atlantic issue which affected a lot of the connectivity between the US and EU and other providers were affected. In our case we had to route around Telia which seemed to be the link that was having the most issues and we're getting more info from them to see what happened and when it will be safe to add them back into the mix. Thanks, Moisey Cofounder DigitalOcean

16 points by ibarrajo 12y ago

nodesocket 12y ago

As a side effect, anybody else getting intermittent false alarms from Pingdom?

peterwaller 12y ago
You're not the only one.
- thehodge 12y ago
  I had this issue on the new beeper system, I switched back to the old system and have had no issues since.
bobfunk 12y ago
yeah, got a bunch - guess they're not strictly speaking false, it's just not something most of us can do anything about...

diorray 12y ago

It's not Digital Ocean's fault. It seems like CloudFlare, Twitter and Facebook's EU servers are down too..

sampo 12y ago
I am in Europe, and I can reach Facebook's EU servers just all right :-).
- deleted 12y ago
  [deleted]
- deleted 12y ago
  [deleted]

carlio 12y ago

A bit of tracerouting and some twitter searching[0] suggests that the issue likes with Telia.net. There are some unconfirmed suggestions that a cable was severed[1], which I'm not sure I believe yet but certainly any packet going via telia.net is having a bad time. Edit: Even CloudFlare are reporting the cable cut[2] so... gosh. [0] https://twitter.com/search?q=telia&src=typd https://twitter.com/search?q=telia&src=typd [1] https://twitter.com/CurseNetworkSys/status/468493477542494208 https://twitter.com/CurseNetworkSys/status/46849347754249420... [2] https://twitter.com/CloudFlare/status/468493714134822912 https://twitter.com/CloudFlare/status/468493714134822912

abritishguy 12y ago

Transatlantic cable has been cut, lots of sites affected.

nodesocket 12y ago

Akamai has a nice visualization map. Look at all that red in Europe. http://www.akamai.com/html/technology/dataviz1.html http://www.akamai.com/html/technology/dataviz1.html

deleted 12y ago

[deleted]

ccozan 12y ago

In Germany we felt it for about 15mins, now the routing is adjusted and we are back to normal. This is a nice reminder that the Internet was designed to resist to such disturbances.

lucaspiller 12y ago

Related discussion: https://news.ycombinator.com/item?id=7769587 https://news.ycombinator.com/item?id=7769587 From the UK everything seems to be working normal now, including access to my instance in AMS1.

ultrasaurus 12y ago

At PagerDuty we saw our alerting volume double between 19:50 and 20:00 UTC (about 90 minutes ago).

raiyu 12y ago

It looks like from our end that there was a large trans-atlantic issue which affected a lot of the connectivity between the US and EU and other providers were affected. In our case we had to route around Telia which seemed to be the link that was having the most issues and we're getting more info from them to see what happened and when it will be safe to add them back into the mix. Thanks, Moisey Cofounder DigitalOcean