4 ms·

> How do we calculate Uptime? > Our Uptime calculation is based on the percentage of successful requests we serve through our web, API, and Git client interfac

by cle 7y ago

> How do we calculate Uptime? > Our Uptime calculation is based on the percentage of successful requests we serve through our web, API, and Git client interfaces. Just curious, how do they measure this? What is the actual calculation?

dorfsmay 7y ago
Same here. They obviously don't have beacons on the client side, I wonder if it's based on statistics, at this time of the day on a Tuesday we should be getting x requests but are getting only x/n.
- heydabop 7y ago
  > They obviously don't have beacons on the client side Says who?
  - dorfsmay 7y ago
    Client side beacons would have to be implemented in the git client, and making parallel requests. I've seen no evidence of it myself, and I'd think people would freak out if found inside git's source.
Benjammer 7y ago
What do you mean? All modern enterprise analytics/monitoring solutions are going to be able to give you some kind of top-level "request success rate" metric. I assume they mostly just lean into whatever monitoring tooling they have set up. What kind of "calculation" are you imagining here? Like a very specific SRE formula for availability windows or something?
- bradstewart 7y ago
  Depending on what part of the system is down, how do you know you even got a request to mark as failed?
  - MetalMatze 7y ago
    You want to start measuring the closet to your users. In most cases that would be some sort of load balancer. I don't think there's much we can do without going to the client side.
- a012 7y ago
  It means their SLA bases on non 5xx and connection errors?
  - Benjammer 7y ago
    I mean that seems like a question to ask an account rep? I'm sure it's also probably not a hard and fast rule for every single customer, hence the ambiguity in the general language.
allanbreyes 7y ago
> What is the actual calculation? Not answering this directly, but the paper Meaningful Availability [0] released recently really changed my opinion on how to calculate and visualize availability. There's a discussion on HN as well [1]. [0]: https://www.usenix.org/system/files/nsdi20spring_hauer_prepub.pdf https://www.usenix.org/system/files/nsdi20spring_hauer_prepu... [1]: https://news.ycombinator.com/item?id=22424173 https://news.ycombinator.com/item?id=22424173
- thanatropism 7y ago
  That was insightful to some point. But, of course, the relevant metric is "expected availability" -- before I decide to go on the service; therefore, not the same as "customers served". If I have to think about downtime then my experience is degraded; moreso if I have to delay and batch planned interactions (all of which will later count as successful!) [Edit: To the point: a high rate of randomly-timed failures is a kind of degraded experience, but not as critical as blocky patches of downtime. A 1% rate of randomly-timed failures is much much much preferred than having the service go out three straight days every February.] Also: uptime is not the same as "customer delight". It's all about time.
  - allanbreyes 7y ago
    > a high rate of randomly-timed failures is a kind of degraded experience, but not as critical as blocky patches of downtime Do you think that's an accurate generalization for all software and business contexts? I think a novel insight about the paper is that windowed user uptime is able to visualize the differences. (See Figure 20 from the paper.)
    - johannes1234321 7y ago
      It certainly is not - a trade might not care about 99% of the time, but the exact moment they want to do a trade the system must work. Whereas of some GitHub request fails and I retry it's a minor annoyance, but in most cases I won't even know whether that was GitHub's flaw, my local system or some networking in between.
jws 7y ago
Our Uptime calculation is based on the percentage of successful requests we serve through our web, API, and Git client interfaces. So when a customer finds a broken service it is in their financial best interest to repeatedly hammer the broken service and drive down the uptime calculation to trigger their rebate. Just an observation, not a suggestion. I’d fire any customer I found doing this.

by cle 7y ago

dorfsmay 7y ago

Same here. They obviously don't have beacons on the client side, I wonder if it's based on statistics, at this time of the day on a Tuesday we should be getting x requests but are getting only x/n.

heydabop 7y ago
> They obviously don't have beacons on the client side Says who?
- dorfsmay 7y ago
  Client side beacons would have to be implemented in the git client, and making parallel requests. I've seen no evidence of it myself, and I'd think people would freak out if found inside git's source.

Benjammer 7y ago

What do you mean? All modern enterprise analytics/monitoring solutions are going to be able to give you some kind of top-level "request success rate" metric. I assume they mostly just lean into whatever monitoring tooling they have set up. What kind of "calculation" are you imagining here? Like a very specific SRE formula for availability windows or something?

bradstewart 7y ago
Depending on what part of the system is down, how do you know you even got a request to mark as failed?
- MetalMatze 7y ago
  You want to start measuring the closet to your users. In most cases that would be some sort of load balancer. I don't think there's much we can do without going to the client side.
a012 7y ago
It means their SLA bases on non 5xx and connection errors?
- Benjammer 7y ago
  I mean that seems like a question to ask an account rep? I'm sure it's also probably not a hard and fast rule for every single customer, hence the ambiguity in the general language.

allanbreyes 7y ago

> What is the actual calculation? Not answering this directly, but the paper Meaningful Availability [0] released recently really changed my opinion on how to calculate and visualize availability. There's a discussion on HN as well [1]. [0]: https://www.usenix.org/system/files/nsdi20spring_hauer_prepub.pdf https://www.usenix.org/system/files/nsdi20spring_hauer_prepu... [1]: https://news.ycombinator.com/item?id=22424173 https://news.ycombinator.com/item?id=22424173

thanatropism 7y ago
That was insightful to some point. But, of course, the relevant metric is "expected availability" -- before I decide to go on the service; therefore, not the same as "customers served". If I have to think about downtime then my experience is degraded; moreso if I have to delay and batch planned interactions (all of which will later count as successful!) [Edit: To the point: a high rate of randomly-timed failures is a kind of degraded experience, but not as critical as blocky patches of downtime. A 1% rate of randomly-timed failures is much much much preferred than having the service go out three straight days every February.] Also: uptime is not the same as "customer delight". It's all about time.
- allanbreyes 7y ago
  > a high rate of randomly-timed failures is a kind of degraded experience, but not as critical as blocky patches of downtime Do you think that's an accurate generalization for all software and business contexts? I think a novel insight about the paper is that windowed user uptime is able to visualize the differences. (See Figure 20 from the paper.)
  - johannes1234321 7y ago
    It certainly is not - a trade might not care about 99% of the time, but the exact moment they want to do a trade the system must work. Whereas of some GitHub request fails and I retry it's a minor annoyance, but in most cases I won't even know whether that was GitHub's flaw, my local system or some networking in between.

jws 7y ago

Our Uptime calculation is based on the percentage of successful requests we serve through our web, API, and Git client interfaces. So when a customer finds a broken service it is in their financial best interest to repeatedly hammer the broken service and drive down the uptime calculation to trigger their rebate. Just an observation, not a suggestion. I’d fire any customer I found doing this.