3 ms·
"Uptime for current quarter: 99.97%" AFAICT, this outage has been going on 6 hours and is still showing a ton of services as "Incident" status in the slack sta
by linsomniac 6y ago
"Uptime for current quarter: 99.97%" AFAICT, this outage has been going on 6 hours and is still showing a ton of services as "Incident" status in the slack status page. Seems like uptime could be more like 99.7% and dropping.
But, how do out count downtime of a distributed system? We probably need a new way of discussing availability.
An outage to part of your system might only impact one or a group of users, but to those users you could have been 100% down, effectively, even as other users were not impacted at all.
So maybe a series of user/%age counts? "This quarter 90% of our users/use cases had 100% availability", for example. With the gold standard being 100/100. Maybe numbers for 99, 90, and 50% of users/use cases? IOW: "1% of our users had 95% availability, 10% of users had 99.9, and 50% of users had 100%."
- joshuanapoli 6y agoFor simplicity, you could use the fraction of api calls that our service level objectives. Google/SRE calls it "aggregate availability". https://landing.google.com/sre/sre-book/chapters/embracing-risk/#risk-management_measuring-service-risk_aggregate-availability-equation https://landing.google.com/sre/sre-book/chapters/embracing-r...
- foxtr0t 6y agoYeah, this problem has a pretty obvious solution that is implemented by many teams: find a unit of work that we can measure success and failure over, report an aggregate metric over a specified time interval for availability tracking and plotting: # units successful / total # of units
- a1369209993 6y agoTo be fair, that has problems like counting "do this thing; do that thing" twice as much as "do these four things" rather than vice-versa. Still useful though.