6 ms·

Also, 3 years ago they had an outage in Strasbourg, and the status page was down apparently as a result of the outage. https://news.ycombinator.com/item?id=156

Also, 3 years ago they had an outage in Strasbourg, and the status page was down apparently as a result of the outage. https://news.ycombinator.com/item?id=15661218 https://news.ycombinator.com/item?id=15661218 They are not the only ones though. All too common. Well, it's tricky to set this up properly. The only proper way would be to use external infra for the status page.

lebaux 6y ago
Isn’t that why things like statuspage.io exist, though?
- dolmen 6y ago
  Where is statuspage.io hosted?
  - formerly_proven 6y ago
    In the cloud of course. Why do you ask?
  - cortesoft 6y ago
    I work for a CDN, and we had to change our status page provider once when they became our customer.
  - deleted 6y ago
    [deleted]
kalleboo 6y ago
It's not difficult to make a status page with minimal false negatives. Throw up a server on another host that shows red when it doesn't get a heartbeat. But then instead you end up with false positives. And people will use false positives against you to claim refunds against your SLA. So nobody chooses to make an honest status page.
- jakub_g 6y ago
  Yep. I guess what could be done is a two-tiered status page: automated health check which shows "possible outage, we're investigating" and then a manual update (although some would say it looks lame to say "nah, false positive" which is probably why this setup is rare).
- toast0 6y ago
  As someone who maintained a status page (poorly), I'm sorry on behalf of all status pages. But, they're usually manual affairs because sometimes the system is broken even when the healthcheck looks ok, and sometimes writing the healthcheck is tricky, and always you want the status page disconnected from the rest of the system as much as possible. It is a challenge to get 'update the status page' into the runbook. Especially for runbooks you don't review often (like the one for the building is on fire, probably). Luckily my status page was not quite public; we could show a note when people were trying to write a customer service email in the app; if you forget to update that, you get more email, but nobody posts the system is down and the status page says everything is ok.

lebaux 6y ago

Isn’t that why things like statuspage.io exist, though?

dolmen 6y ago
Where is statuspage.io hosted?
- formerly_proven 6y ago
  In the cloud of course. Why do you ask?
- cortesoft 6y ago
  I work for a CDN, and we had to change our status page provider once when they became our customer.
- deleted 6y ago
  [deleted]

kalleboo 6y ago

It's not difficult to make a status page with minimal false negatives. Throw up a server on another host that shows red when it doesn't get a heartbeat. But then instead you end up with false positives. And people will use false positives against you to claim refunds against your SLA. So nobody chooses to make an honest status page.

jakub_g 6y ago
Yep. I guess what could be done is a two-tiered status page: automated health check which shows "possible outage, we're investigating" and then a manual update (although some would say it looks lame to say "nah, false positive" which is probably why this setup is rare).
toast0 6y ago
As someone who maintained a status page (poorly), I'm sorry on behalf of all status pages. But, they're usually manual affairs because sometimes the system is broken even when the healthcheck looks ok, and sometimes writing the healthcheck is tricky, and always you want the status page disconnected from the rest of the system as much as possible. It is a challenge to get 'update the status page' into the runbook. Especially for runbooks you don't review often (like the one for the building is on fire, probably). Luckily my status page was not quite public; we could show a note when people were trying to write a customer service email in the app; if you forget to update that, you get more email, but nobody posts the system is down and the status page says everything is ok.