5 ms·

DigitalOcean Down?

56 points by geerlingguy 5y ago

geerlingguy 5y ago
Status page [1] not showing anything wrong currently, but I had my own VPS go offline a couple minutes ago, and now the login page for their Cloud UI is also offline with the message: {"error":"Looks like something went wrong. If this continues, please open a support ticket so we can help you out"} Edit: And now all my NYC3 VPSes are going offline at 12:01 US Eastern (just got a bunch of outage emails from my monitoring service). Edit 2: Cloud UI Login page briefly started working at 12:12 US Eastern, but 2FA login step is not completing. One of my servers is reachable, but about 5 others still aren't. Hopefully it's a sign things are coming back... Edit 3: As of 12:16 US Eastern, all my NYC3 VPSes are back online, but Cloud UI login still failing at 2FA step. Edit 4: As of 12:24 US Eastern, it seems everything is back up (for me), but they finally updated their status page with "Multiple Products Down". [1] https://status.digitalocean.com https://status.digitalocean.com
- crossworth 5y ago
  I'm facing the same problem.
- tech5000 5y ago
  Down for me as well. All sites completely offline and showing our CDN provider's Error 522 page on all customer-facing sites. Same error as yours when I tried to log in and see what's up.
- nnf 5y ago
  I'm experiencing the same thing. Can't reach my droplets, and can't sign in to see what's going on. Edit [12:16pm EST]: my Droplets are reachable now, and my web apps are back, but I still can't access the control panel, and the status page hasn't been updated to show any problems.
  - geerlingguy 5y ago
    > the status page hasn't been updated to show any problems. The only thing is 'Spaces' (which I'd never heard of until today), but I'm not sure how Spaces would affect the entire NYC3 data center (plus the cloud.digitalocean.com backend).
    - nnf 5y ago
      The Spaces (Digital Ocean's S3-like object storage service) outage was going on yesterday as well, so that particular part may be unrelated and ongoing.
bithavoc 5y ago
Droplets stopped responding. Edit: Droplets are back in NYC3
aosaigh 5y ago
Just noticed the same. I'm still connected to my VPS over SSH, but HTTP traffic is timing out. Think it's US East only (my European sites are still OK).
james33 5y ago
Looks like a widespread outage. We have servers in multiple regions that are all down and we can't even get into the DO dashboard.
soulchild37 5y ago
Yes, I got noticed from uptime monitoring service, my SaaS are running solely on DigitalOcean ughhhh
marpstar 5y ago
Same. Can't access cloud admin panel. I've got two sites, one on NYC1 and one on NYC3. One is up and running, the other is not.
marpstar 5y ago
It's back up now.
blhack 5y ago
Well that was a shitty 5 minutes!
nickjj 5y ago
All of my NYC hosted sites went down too but they're back up now. I'm not really upset over it and my whole business depends on my sites being up. DO's uptime has been really really good in general. I can't remember the last time everything went down like this in NYC3 over the last ~6 years.
- adventured 5y ago
  Saw the same thing out of NYC. It seemed to only last a few minutes, I didn't bother to investigate further due to the short duration.
NicoJuicy 5y ago
My droplets aren't down, my own status page didn't notice anything. DC Netherlands ( Didn't login yet) Ps. Status page+ checks have no dependency on DO
sillycube 5y ago
Oh, I thought that it's my fault. There were a couple minutes outage for my app.
bovermyer 5y ago
That was a really, really brief downtime. My two sites that were affected were offline for less than five minutes.
- montroser 5y ago
  I would not call five minutes "really, really brief". Could have been worse, but for some applications, five minutes can be an eternity.
  - dsr_ 5y ago
    If your application can't cope with five minutes of downtime, I'm sure you're not relying on a single cloud vendor.
username91 5y ago
My London-hosted site stayed up just fine. DO's uptime has been extremely impressive to me.
tech5000 5y ago
Here is a data point: In the last ~90 minutes the external ping tool we use has shown the site changing between accessible then inaccessible 25 times.
tomxor 5y ago
I know this is not usually a welcome question but... Why is this flagged? It seems like legitimately concerning news many HN readers would want to know about, and I only barely caught it before it was banished from the front page. I really can't see a reason.
ruben81adelaide 5y ago
Yes, I monitor about 1000 servers in Digital Ocean in New York, London and Singapore. Only New York went down for about 1 hour and a couple of hours later the global DNS system in DO went down as well for about 15 min. They initially didn't made any update in their Status page

56 points by geerlingguy 5y ago

Status page [1] not showing anything wrong currently, but I had my own VPS go offline a couple minutes ago, and now the login page for their Cloud UI is also offline with the message: {"error":"Looks like something went wrong. If this continues, please open a support ticket so we can help you out"} Edit: And now all my NYC3 VPSes are going offline at 12:01 US Eastern (just got a bunch of outage emails from my monitoring service). Edit 2: Cloud UI Login page briefly started working at 12:12 US Eastern, but 2FA login step is not completing. One of my servers is reachable, but about 5 others still aren't. Hopefully it's a sign things are coming back... Edit 3: As of 12:16 US Eastern, all my NYC3 VPSes are back online, but Cloud UI login still failing at 2FA step. Edit 4: As of 12:24 US Eastern, it seems everything is back up (for me), but they finally updated their status page with "Multiple Products Down". [1] https://status.digitalocean.com https://status.digitalocean.com

crossworth 5y ago
I'm facing the same problem.
tech5000 5y ago
Down for me as well. All sites completely offline and showing our CDN provider's Error 522 page on all customer-facing sites. Same error as yours when I tried to log in and see what's up.
nnf 5y ago
I'm experiencing the same thing. Can't reach my droplets, and can't sign in to see what's going on. Edit [12:16pm EST]: my Droplets are reachable now, and my web apps are back, but I still can't access the control panel, and the status page hasn't been updated to show any problems.
- geerlingguy 5y ago
  > the status page hasn't been updated to show any problems. The only thing is 'Spaces' (which I'd never heard of until today), but I'm not sure how Spaces would affect the entire NYC3 data center (plus the cloud.digitalocean.com backend).
  - nnf 5y ago
    The Spaces (Digital Ocean's S3-like object storage service) outage was going on yesterday as well, so that particular part may be unrelated and ongoing.

bithavoc 5y ago

Droplets stopped responding. Edit: Droplets are back in NYC3

aosaigh 5y ago

Just noticed the same. I'm still connected to my VPS over SSH, but HTTP traffic is timing out. Think it's US East only (my European sites are still OK).

james33 5y ago

Looks like a widespread outage. We have servers in multiple regions that are all down and we can't even get into the DO dashboard.

soulchild37 5y ago

Yes, I got noticed from uptime monitoring service, my SaaS are running solely on DigitalOcean ughhhh

marpstar 5y ago

Same. Can't access cloud admin panel. I've got two sites, one on NYC1 and one on NYC3. One is up and running, the other is not.

marpstar 5y ago

It's back up now.

blhack 5y ago

Well that was a shitty 5 minutes!

nickjj 5y ago

All of my NYC hosted sites went down too but they're back up now. I'm not really upset over it and my whole business depends on my sites being up. DO's uptime has been really really good in general. I can't remember the last time everything went down like this in NYC3 over the last ~6 years.

adventured 5y ago
Saw the same thing out of NYC. It seemed to only last a few minutes, I didn't bother to investigate further due to the short duration.

NicoJuicy 5y ago

My droplets aren't down, my own status page didn't notice anything. DC Netherlands ( Didn't login yet) Ps. Status page+ checks have no dependency on DO

sillycube 5y ago

Oh, I thought that it's my fault. There were a couple minutes outage for my app.

bovermyer 5y ago

That was a really, really brief downtime. My two sites that were affected were offline for less than five minutes.

montroser 5y ago
I would not call five minutes "really, really brief". Could have been worse, but for some applications, five minutes can be an eternity.
- dsr_ 5y ago
  If your application can't cope with five minutes of downtime, I'm sure you're not relying on a single cloud vendor.

username91 5y ago

My London-hosted site stayed up just fine. DO's uptime has been extremely impressive to me.

tech5000 5y ago

Here is a data point: In the last ~90 minutes the external ping tool we use has shown the site changing between accessible then inaccessible 25 times.

tomxor 5y ago

I know this is not usually a welcome question but... Why is this flagged? It seems like legitimately concerning news many HN readers would want to know about, and I only barely caught it before it was banished from the front page. I really can't see a reason.

ruben81adelaide 5y ago

Yes, I monitor about 1000 servers in Digital Ocean in New York, London and Singapore. Only New York went down for about 1 hour and a couple of hours later the global DNS system in DO went down as well for about 15 min. They initially didn't made any update in their Status page