7 ms·
Digital Ocean NYC2 Droplets Down
And counting. Second time in about a week and during prime time hours. If it doesn't get up quickly or if happens again to NYC2 in the next few weeks may be time to reconsider DO for hosting production sites. I love them, and glitches happen, but twice in a row clients have a hard time swallowing.
- n0nenet 12y agoI have to agree with you on this one.
- marcusr 12y agoIt seems to be http (perhaps load balancers) rather than the instances being down - I'm ssh-ed into a droplet and everything is working fine, except remote HTTP connections.
- iurisilvio 12y agoI ssh-ed into some droplets, but others are completely inacessible outside DO. The internal network works fine, so it is just a several network outage. Again.
- publicfig 12y agoI'm struggling to ssh into droplets, but it seemed like when I was ssh'd in, I didn't have an issue at all.
- mikeash 12y agoIt seems to be intermittently up and down. So your connections all die, and you can't reconnect for a while, then it comes back and everything looks good for a bit until it dies again.
- heavymark 12y agoSame here. Since 1:24p, Pingdom has reported our sites going up and down about 16 times.
- toomuchtodo 12y agoWhat happens when you try to execute outbound web requests from the instance? http://netalyzr.icsi.berkeley.edu/cli.html http://netalyzr.icsi.berkeley.edu/cli.html
- JohnTHaller 12y agoI've been unable to SSH in for an hour and a half. But I expected this when I signed up (cheap = cheap) so have backups running that were switched to.
- mbreedlove 12y agoI'd love to know what's going on. My website hosted on DO, michaelbreedlove.net, is unavailable. I also have a few services running on subdomains of the same DO VPS instance; they're all accessible. I also have SSH access. They seem to be having some serious HTTP routing issues.
- publicfig 12y agoI just created a droplet today, and was trying for the life of me to figure out what I would have done so early on in setting it up to cause it to become unresponsive. Glad to see it's not just me. It's not really a great sign though, as I was hoping to use this fairly frequently. Is this a common occurrence, or do I just have bad luck? If so, I'll definitely have to find another solution.
- zhodge 12y agoI'm not too qualified to comment on the hosting market, but I do want to vouch for DigitalOcean as I've never had uptime issues with them until now. They also have always had timely and helpful support and their pricing is obviously competitive. That being said, near 100% uptime is really desired in this space and I've been scratching my head for the past hour or so wondering why I couldn't connect to my mongo process running on my droplet...maybe I should have tested DO sooner, but the fact that only mongo seemed to be failing (and not postgres for example) made me think I had client and not server issues.
- gingerlime 12y agoI've been using Linode for many years, and DO for some time now (a year? not sure, since I can't even login to their control panel right now and check). I've seen issues on both providers, and dealt with support of both not too infrequently. I'd still say Linode is miles ahead in terms of expertise and responsiveness. Most of my servers are on Linode for this reason, and only a few on DO. As a customer I enjoy this competition and would like to keep both players compete - because I'm benefiting from it. So I'm likely to stick to both. But I would still recommend Linode more easily than I would DO.
- zhodge 12y agoThanks for your input. I might be switching over as a result of the recent downtime and the support Linode has been receiving on this thread. Unfortunately, especially for non-hobbyist apps and services, this is an area demanding fickleness.
- 12y ago
- deleted 12y ago[deleted]
- jqueryin 12y agoHaving previously left Linode for Digital Ocean due to lower pricing and SSDs, I can safely say Linode both dropped their prices and matched the DO plans with SSDs included. If you want to jump ship, that's the route I'd go for a comparable. I still love DO, but downtime is a no-no when it comes to VPSes.
- ramatgan 12y agoIn the process of switching over now.
- JohnTHaller 12y agoWhen you charge so little and the SLA is just credit during the time of the outtage (pennies), downtime is pretty much a given. It always surprises me that this surprises people.
- uptown 12y agoHere's their status panel: https://status.digitalocean.com/ https://status.digitalocean.com/
- deleted 12y ago[deleted]
- dangoldin 12y agoOne of my instances on NY2 is fine and the other one is sporadically accessible via SSH. Hope they figure this one out soon.
- jcampbell1 12y agoLinode's new offering probably the better choice these days. I get the feeling that DO is just overwhelmed with problems related to their hyper growth - 10% per month on a huge base. http://trends.netcraft.com/www.digitalocean.com http://trends.netcraft.com/www.digitalocean.com
- mctDarren 12y agoThey've done so well to this point keeping up with the growth curve. Hope they don't become crippled by success.
- bmoresbest55 12y agoThis is a little ridiculous. I hope not to have this problem again. Second strike her. Unfortunately.
- potomushto 12y agohttps://cloud.digitalocean.com/ https://cloud.digitalocean.com/ now is unavailable too.
- alokyadav15 12y agoYeah , NYC2 seems affected https://twitter.com/digitalocean/status/494170340973887488 https://twitter.com/digitalocean/status/494170340973887488
- kl4m 12y agoTheir status page indicates "We have identified a slow connection to our NYC2 data center". Assuming they have redundant connections, this seems like a big understatement for being down for 1 hour and going. Even their control panel is down for me. ( http://i.imgur.com/3ltg00o.png http://i.imgur.com/3ltg00o.png )
- bigredtech 12y agoI'm a big fan of Linode. Have had one of their mid-range servers for about 2 years now and have had no troubles. When it came time to upgrading, or any other questions I had, their support staff is 1) quick, 2) knowledgable, 3) thorough & helpful. I've used DO in the past for quick spin-ups, and they're great for that. But for long running applications I'd stay away. Was actually considering DO for a new project, but this kind of tipped the scales the other way. If you're looking to switch, I think Linode gives you the best bang and reliability for your buck. Have also used Rackspace for a previous startup. A bit more pricey, but also a great service if you're considering switching.
- meritt 12y agoLinode can't silo resources on a per VPS basis and constantly will harass you for over-utilizing your share of arbitrarily enforced limits. Just a heads up if you actually utilize your instances.
- harpastum 12y agoAre you talking about the "Linode Alert" emails? Those are user-configurable — in the email, they specifically state "This is not meant as a warning or a representation that you are misusing your resources. We encourage you to modify the thresholds based on your own individual needs." If that's not what you're talking about, I apologize, but I've been using Linode for years (often with very compute-intensive projects) and I haven't seen what you're talking about.
- jarin 12y agoJust out of pure laziness, I use the CPU and disk access emails as makeshift notifications for when my backups run.
- deleted 12y ago[deleted]
- opendais 12y agoReally? I find that odd. I do load testing on their instances all the time. which maxes out cpu/disk to give me an idea of performance changes for stuff I have high throughput on. Hell, I've gotten DDoS'd for like 2 Gbps and they were like "Oh, ya, its fine." So, can you explain more about what happened?
- Sir_Cmpwn 12y agoI am getting pretty sick of it, too. I might go back to Linode. I was already upset that they dropped Arch Linux support.
- opendais 12y agoLinode has price matched them so its not a price issue anymore. [e.g. $10 Linode is roughly the same as $10 DO] Glad to see the ProjectWonderful ads providing something, even if its small btw. :)
- JohnTHaller 12y agoYou should probably reconsider hosting production sites on super-cheap VPSes with a 1:1 SLA refund. They're going to go down. And, when they do, you'll get the 7/10ths of a cent refund per hour promised for your $5 instance (1.4 cents per hour for $10, 2.8 cents per hour for $20, etc). But if you depend on a site for your business, ask yourself how much it costs you per hour of downtime and plan appropriately. For context: I'm a happy Digital Ocean customer (as well as Rackspace and others). I like their services and consider them a great value for the money. I've also gotten multiple developer friends into using them. But I don't let them be a single point of failure for anything critical.
- kbar13 12y agodowntime is an issue no matter how much you pay your infrastructure provider. You should account for downtime in your application and your infra by taking advantage of low-hanging high availability fruit. Things like saltstack (saltstack.org) make this mega-easy.
- JohnTHaller 12y agoIt's easier for apps since you can build in failover support based on multiple servers pretty easily (I just implemented this in our 'live' installers recently on Windows). It's a bit harder at the website level for an average user, though. I know far too many people that have a single point of failure website on a super-cheap VPS that's just down whenever the VPS is down. Most of them would have no idea how to implement a load balancer and failover support. Though I could probably get them into something like CloudFare or similar.
- neom 12y agowe've had an ongoing issue with a juniper fiber core in nyc2 it seems. We've engaged juniper to help us, I don't expect this to be an issue again.
- deleted 12y ago[deleted]
- alokyadav15 12y agoits working now , atleast for me .
- JohnTHaller 12y agoI'm back up as of 2:59pm EDT, so I was down for about 1:35. This is the only outtage longer than a few minutes I've had with DO thusfar.
- larrys 12y ago"but twice in a row clients have a hard time swallowing." May I ask what your clients are paying? Maybe the answer is to charge them more instead of giving them a low price (an assumption on my part of course which is why I ask what you are getting for this) and some implicit guarantee of uptime. For example we had some customers some time ago who were pooled on a shared IP address for sending SMTP service. Occasionally they would experience problems with mail reputation on that shared IP. If they complained about that would simply explain that that was the drawback of a shared IP and that they could always upgrade to a service level where they got a unique IP and could establish their own reputation.
- Donzo 12y agoI've had great experiences with them. I recommended DO to everyone. Now I'm tempted to drive to New York and punch them each in the face. PEAK downtime. EDIT: $160 refund. Maybe just a Charlie horse.
- opendais 12y agoFolks, I strongly, strongly suggest if you use Cloud/VPS providers to look into something like Cassandra, Galera, Mongo, etc. that can do WAN replication and using 3 DCs. Yes it costs money but things like this are avoidable. Every single Datacenter on the planet has downtime and/or performance issues from time to time. Hell, one of the places I have a dedi at had its network performance drop by 50% for about 15 minutes while they did emergency maintenance. Things like this are going to happen as long as you depend on one provider.
- rrss1122 12y agoWe're getting the same thing. I don't care how cheap it is or how nice the customer service is, but if they are not up 99% of the time they will not be my web host.
- wglb 12y agoEr, no. My droplets are up. Never went down today. Also, please don't post that things are down here on HN. This is not a support forum. Not for DO, not for AWS.
- iurisilvio 12y agoYou came here after it was fixed. It is fine now. Also, it was just NYC2, maybe only part of it.
- wglb 12y agoMine are on NYC2. Terminal sessions up all day. On three of them. Check http://downforeveryoneorjustme.com http://downforeveryoneorjustme.com instead of HN.
- hadoukenio 12y agoWhy do people have their eggs in one basket? More specifically, whenever hosting comes up on HN, it's always DO vs Linode. Why not Zoidberg^Wboth? In the age of DevOps, it's not hard to configure different providers. I have been using Linode and DO for a while now. I noticed DO go down last week, checked my site, and it was UP because it was being served via Linode. I flinch every time I hear someone say to me that their service has redundancy because they use multiple Amazon regions. Multiple providers people! And preferably multiple providers in multiple states, paid for by mutually exclusive credit cards! Redundancy doesn't mean having multiple hosts by the same provider, within the same data center, within the same rack, provided by the same power strip. tl;dr: Plan for outages.
- opendais 12y agoIt is always DO vs. Linode because both are relatively stable, relatively good for their price points. Tbh, the only real reason I use Linode over DO is the fact I've seen higher restarts on my DO nodes and the fact I've attracted DDoS's at a size Linode shrugs about.
- anderspetersson 12y agoHow do you do this on the DNS layer? Very short TTL and switching on failure detection? Do you use both providers at the same time when both are up via DNS round robin or similar?
- hadoukenio 12y agoThere's nothing stopping you from adding multiple A records for a domain. In fact, that's what a lot of people do: dig a www.google.com www.google.com. 300 IN A 74.125.237.178 www.google.com. 300 IN A 74.125.237.177 www.google.com. 300 IN A 74.125.237.176 www.google.com. 300 IN A 74.125.237.179 www.google.com. 300 IN A 74.125.237.180 Browsers these days will resolve and try each one, on a first-come-first-served basis. If one fails, it will try the request on the next server.
- stevekemp 12y ago