5 ms·
Heroku down
- tferris 15y ago> ISSUE: We're experiencing a widespread outage affecting our HTTP routing doesn't sound good
- deleted 15y ago[deleted]
- dedene 15y agoIndeed, along with a whole lot of other web apps relying on Heroku.
- aen 15y agoThe status page (https://status.heroku.com/ https://status.heroku.com/) is also really slow now.
- alexchamberlain 15y agoProbably not designed for the HN effect.
- StavrosK 15y agoIt's probably not running on Heroku (imagine the irony).
- imbriaco 15y agoIt's definitely not running on Heroku.
- dotty 15y agoWhy on earth would anyone ever run a status page for a platform - used when the platform goes down - on the platform itself? :|
- alexchamberlain 15y agoIt's generally good practice to run your monitoring website from a different datacenter (or at least a different server).
- StavrosK 15y agoI meant "the irony of running a status page that's reporting when a platform is down on the platform itself", not the irony that heroku isn't running its status page on itself.
- latchkey 15y agoI was wondering why http://intercom.io http://intercom.io was down and now I come here and see this at the top. Must be it. Update, they seem back now, but their site was throwing errors a minute ago.
- purephase 15y agoInteresting. I was just considering them over AWS for hosting. Current users, how reliable is Heroku? While I appreciate openess, looking over that status page does not inspire a lot of confidence.
- robin_reala 15y agoJust checked: 99.94% for our main site, 99.93% for a promo site.
- Argorak 15y agoAt the size of Heroku, the probability of a server outage per day is actually quite high. As long as the errors are just "one of our machines has gone down", its an interesting information for everyone on the machine, but it doesn't say a lot about the reliability at all. Major outages are rare and I am quite pleased.
- purephase 15y agoThanks.
- 4ad 15y agoHeroku is just a layer over AWS. Major disruptions have been caused so far only by AWS failures.
- jamiemill 15y agoIn my personal experience, according to pingdom my Heroku app is significantly less reliable than my plain EC2 apps: https://twitter.com/#!/jamiemill/status/154566469479444480/photo/1 https://twitter.com/#!/jamiemill/status/154566469479444480/p... For ref, the heroku app is rails, 2x dynos and a shared DB with minimal traffic. The others are PHP/apache apps, one using RDS, on plain EC2. Heroku is at least 5 layers extra to go wrong on top of EC2. On the one hand I feel that each layer adds to the probability of failure. But I'd hope that one of the benefits of Heroku would be extra redundancy and failure handling, though my limited personal experience hasn't been that way so far. EDIT: Regarding openness, I certainly feel that every single outage is admitted by Heroku, whereas I feel Amazon's status page has not listed a few outages I've suffered. Again personal feeling, not evidence.
- dedene 15y agoThey're up again?
- jaryd 15y agoLets hope the simplicity of 'git push heroku' hasn't distracted most developers (and customers of Heroku) from Werner Vogels' core tenet of administration: Everything fails all the time. (http://technocation.org/files/videos/original/mysqlconf2008/2008_04_15_amazonKeynote.wmv http://technocation.org/files/videos/original/mysqlconf2008/...) This might separate the men from the boys (excuse the idiom) to see who has adequately prepared a DR/redundancy plan for when the cloud fails. Edit: Improved link to Vogels' MySQL keynote from which the quote originated.
- dasil003 15y agoPersonally I don't think it makes sense to build in redundancy if you're using Heroku, maybe a fallback to a simple offsite "we're down page" but that's it. My reasoning is that if you are on Heroku you should be small and leveraging Heroku's management to the fullest. If you are big you are paying an incredible premium to be on Heroku (thought experiment: when does the premium start costing more than a full-time systems engineer). There is no Heroku equivalent from another company, at least nothing that's really turnkey in the same way, so if you build redundancy you're throwing away the benefit of Heroku's PaaS. So in my mind if you need redundancy you need to be building your own system stack. Whether it's co-located, dedicated, or cloud-based is a separate issue because it's more or less commoditized. You build your system on a portable platform and then redundancy is just a proportional amount of extra work.
- jaryd 15y agoThanks for the thoughtful reply. I think that you raise some valid points here. Regarding "If you are big you are paying an incredible premium to be on Heroku": Sure, Heroku is great (or cheap) for development and small-time deployment, however it is likely true that people get locked into it and (hopefully) grow quickly before they can scale away from a PaaS solution. In these cases, it is imperative to consider the marginal cost of alternative/off-site failover vs. the cost of downtime. This is what I meant by "separate the men from the boys". Those guys who are paying the incredible premium to continue to host on Heroku likely -know- that they are doing so, and have made that decision consciously. While the situation is different, I am reminded of when AWS went down awhile back and the sites that had automated failover to alternative clouds really stood out. The bottom line is that no one solution is ever enough. Tiny apps can likely get away with unexpected//unplanned downtime, however it ultimately delegitimizes them in the eyes of their users (read: customers), and that is universally bad.
- numbdemon 15y agosome services are accessable now ， but some not
- ChrisAnn 15y agoExcuse me, but why is this top of Hacker News? Services go down sometimes, especially cloud ones. No need to make it harder for them by voting it to the top of a major tech news site... edit - spelling
- tferris 15y agomaybe because a lot of of us are hosting their web services on heroku and we don't check mails and monitoring services every minute (but HN)?
- rdl 15y ago10-20% of apps developed by people on Hacker News probably use Heroku. It's not just a random service people use; if there is an outage, service operators will need to field end user inquiries, restore service, etc. hn is probably not a replacement for a good offsite monitoring service, but for me, I was browsing "why does my friend's new app not work; is he updating it live?", switched tabs to hn, saw "heroku down", and all was clear.
- melling 15y agoI think 90% of statistics are made up on the spot.
- wyuenho 15y agoAround 18% of the people responded to the poll said they are hosting on a PasS, which I assume is mostly on Heroku. https://news.ycombinator.com/item?id=3466168 https://news.ycombinator.com/item?id=3466168
- melling 15y agoI did not respond but I use GAE. I think I felt better when I knew the number was just made up.
- 15y ago
- instakill 15y agoIt's up for me
- jbeynon 15y agoIf a Heroku application's DNS is setup correctly then you can usually avoid problems like this on the new Heroku stack (CEDAR). It's important to reread the documentation on using custom domains but also be VERY aware of using naked domains (mysite.com) in your application. This isn't just a Heroku problem, it's a DNS problem. Heroku have recently had a spate of routing issues (usually DDoS attacks) which would have been largely negated by correct DNS setup and either avoiding naked domains or using a DNS host that lets you cname a naked domain to a host.
- kilburn 15y agoCould you please elaborate on why exactly using (non-CNAMEd) "naked domains" is a problem? What do "naked domains" and DDoS attacks have to do with each other?
- glenngillen 15y agoDNS only allows you to put an IP addresses on the apex (read: bare/naked/etc) domain A record. That means in the case of a DDoS or other problem affecting one of those IPs you run the risk of having degraded redundancy. Some people unfortunately only put one IP in, which means you're only one machine away from going offline. A long way from what you'd want from the cloud. DNSimple have created an ALIAS record type which gets around this problem nicely, and Route 53 from Amazon takes a similar approach.