4 ms·

I say this every time Slack is down, but they just seem so shady to me. Nobody can connect right now, and their status site says "100% uptime in the last quart

I say this every time Slack is down, but they just seem so shady to me. Nobody can connect right now, and their status site says "100% uptime in the last quarter". Maybe it's close to 100%, but it ain't 100%. I think we should push for a metric where "up" means 100% of people that want to use the service are able to use the service. If 1% of users can't send messages, then that should count as a full-blown outage and should start counting against whatever SLA they advertise. The underlying problem here is that apparently everyone lies about uptime, so if you don't, that looks bad to potential customers. I fear that we will have to push for some legal regulation if we want accurate data, and ... people will probably be opposed to that.

de_Selby 6y ago
They should at least update the status site to reflect issues currently happening. I was wondering why the link from a Jira wasn't opening in slack, the page eventually timed out and gave me a link to status.slack.com where it told me everything was peachy. Cue me wasting time trying it again because apparently there was no issue with slack..
nkassis 6y ago
You'll just end up with no SLA or pay a hefty amount to use services because that's an impossible standard to support for any service of a size like this.
- scrose 6y ago
  Isn't this the problem? Companies like Slack set SLA's that they only meet by lying about their uptime. It's as good as having no SLA, except you're likely paying a premium based on the SLA they set.
- jrockway 6y ago
  I'm not demanding 100% uptime, I'm asking that they say "99.94% uptime" when there has been an outage. Honestly, I could live with a 99.50% SLA, if that's what it really was. After today's probably full-day outage, they'd just have to be extra careful for the rest of the year (or pay me money). Kind of sucks when it's 1/4 that you blow your year's SLA budget though.
derefr 6y ago
> I think we should push for a metric where "up" means 100% of people that want to use the service are able to use the service. I mean, that’s nice to say, but how do you measure/prove it? Certainly, having the SLAed party check themselves is silly. But what are the other options? If it was up to the customer, customers could make up faults to get free service. (Since it’d be up to the customer to prove, and customers are generally less technical than vendors, you’d have to expect/accept very non-technical — and thus non-evidentiary! — forms of “proof”, e.g. “I dunno, we weren’t able to reach it today.” Things that could have just as well been their own ISP, or even operator error on their side.) IMHO, contractual SLAs should be based on the checks of some agreed-upon neutral-third-party auditor (e.g. any of the many status/uptime monitoring services.) If the third party says the service is up, it’s up in SLA terms; if the third party says the service is down, it’s down in SLA terms. (And, of course, if the third party themselves go down, or experience connectivity issues that cause them to see false correlated failures among many services, that should be explicitly written into the SLA as a condition where the customer isn’t going to get a remedial award against the SLA, even if the SLAed service does go down during that time. If the Internet backbone falls over, that’s the equivalent of what insurance providers call an “act of God.”) But in a neutral-third-party observer setup, you aren’t going to get 100% coverage for customer-seen problems. An uptime service isn’t going to see the service the way every single customer does. Only the way one particular customer would. So it’s not going to notice these spurious some-customers-see-it-some-don’t faults. So, again: what kind of input would feed this hypothetical “100% of customers are being served successfully” metric? ETA: maybe you could get closer to this ideal by ensuring that the monitoring service 1. is effectively running a full integration test suite, not just hitting trivial APIs; and 2. if gradual-rollout experiments ala “hash the user’s ID to land them in an experiment hash-ring position, and assign feature flags to sections of the hash ring” are in use by the SLAed service, then the monitoring service should be given N different “probe users” that together cover the complete hash-ring of possible generated-feature-flag combinations. Or given special keys that get randomly assigned a different combination of feature-flags every time they’re used.
swsieber 6y ago
Some companies do this, though probably not publishing data. Any customer downtime is treated the same - for one, for many, for all (in theory, ha ha). But they take it pretty seriously.
teraflop 6y ago
> If 1% of users can't send messages, then that should count as a full-blown outage and should start counting against whatever SLA they advertise. Google published a paper last year describing this approach to measuring uptime: https://blog.acolyer.org/2020/02/26/meaningful-availability/ https://blog.acolyer.org/2020/02/26/meaningful-availability/ The idea is to define availability as "the probability that the site 'appeared' to be down for a random user, averaged over a time window of size w". You can choose a particular value of w and look at trends over time, or you can plot availability as a function of w to understand patterns of downtime.
gwright 6y ago
Seems silly to worry about quarterly stats several hours into an outage. The most obvious explanation is quarterly stats aren't generated in real-time -- which isn't "shady" to me.
helper 6y ago
That number is almost certainly updated manually. Check back tomorrow and see what it says. If you look at the history page you can see its not 100% for every month: https://status.slack.com/calendar https://status.slack.com/calendar
- helper 6y ago
  (2 days later) The status site now shows %99.9 uptime for the quarter.

de_Selby 6y ago

They should at least update the status site to reflect issues currently happening. I was wondering why the link from a Jira wasn't opening in slack, the page eventually timed out and gave me a link to status.slack.com where it told me everything was peachy. Cue me wasting time trying it again because apparently there was no issue with slack..

nkassis 6y ago

You'll just end up with no SLA or pay a hefty amount to use services because that's an impossible standard to support for any service of a size like this.

scrose 6y ago
Isn't this the problem? Companies like Slack set SLA's that they only meet by lying about their uptime. It's as good as having no SLA, except you're likely paying a premium based on the SLA they set.
jrockway 6y ago
I'm not demanding 100% uptime, I'm asking that they say "99.94% uptime" when there has been an outage. Honestly, I could live with a 99.50% SLA, if that's what it really was. After today's probably full-day outage, they'd just have to be extra careful for the rest of the year (or pay me money). Kind of sucks when it's 1/4 that you blow your year's SLA budget though.

derefr 6y ago

> I think we should push for a metric where "up" means 100% of people that want to use the service are able to use the service. I mean, that’s nice to say, but how do you measure/prove it? Certainly, having the SLAed party check themselves is silly. But what are the other options? If it was up to the customer, customers could make up faults to get free service. (Since it’d be up to the customer to prove, and customers are generally less technical than vendors, you’d have to expect/accept very non-technical — and thus non-evidentiary! — forms of “proof”, e.g. “I dunno, we weren’t able to reach it today.” Things that could have just as well been their own ISP, or even operator error on their side.) IMHO, contractual SLAs should be based on the checks of some agreed-upon neutral-third-party auditor (e.g. any of the many status/uptime monitoring services.) If the third party says the service is up, it’s up in SLA terms; if the third party says the service is down, it’s down in SLA terms. (And, of course, if the third party themselves go down, or experience connectivity issues that cause them to see false correlated failures among many services, that should be explicitly written into the SLA as a condition where the customer isn’t going to get a remedial award against the SLA, even if the SLAed service does go down during that time. If the Internet backbone falls over, that’s the equivalent of what insurance providers call an “act of God.”) But in a neutral-third-party observer setup, you aren’t going to get 100% coverage for customer-seen problems. An uptime service isn’t going to see the service the way every single customer does. Only the way one particular customer would. So it’s not going to notice these spurious some-customers-see-it-some-don’t faults. So, again: what kind of input would feed this hypothetical “100% of customers are being served successfully” metric? ETA: maybe you could get closer to this ideal by ensuring that the monitoring service 1. is effectively running a full integration test suite, not just hitting trivial APIs; and 2. if gradual-rollout experiments ala “hash the user’s ID to land them in an experiment hash-ring position, and assign feature flags to sections of the hash ring” are in use by the SLAed service, then the monitoring service should be given N different “probe users” that together cover the complete hash-ring of possible generated-feature-flag combinations. Or given special keys that get randomly assigned a different combination of feature-flags every time they’re used.

swsieber 6y ago

Some companies do this, though probably not publishing data. Any customer downtime is treated the same - for one, for many, for all (in theory, ha ha). But they take it pretty seriously.

teraflop 6y ago

> If 1% of users can't send messages, then that should count as a full-blown outage and should start counting against whatever SLA they advertise. Google published a paper last year describing this approach to measuring uptime: https://blog.acolyer.org/2020/02/26/meaningful-availability/ https://blog.acolyer.org/2020/02/26/meaningful-availability/ The idea is to define availability as "the probability that the site 'appeared' to be down for a random user, averaged over a time window of size w". You can choose a particular value of w and look at trends over time, or you can plot availability as a function of w to understand patterns of downtime.

gwright 6y ago

Seems silly to worry about quarterly stats several hours into an outage. The most obvious explanation is quarterly stats aren't generated in real-time -- which isn't "shady" to me.

helper 6y ago

That number is almost certainly updated manually. Check back tomorrow and see what it says. If you look at the history page you can see its not 100% for every month: https://status.slack.com/calendar https://status.slack.com/calendar

helper 6y ago
(2 days later) The status site now shows %99.9 uptime for the quarter.