4 ms·

How is github being down losing you guys money?

dewey 2mo ago
If you pay developers x money / day and one of their core tools is down for n hours during the day and they spend their money on HN instead that's pretty straight forward to calculate.
- parthdesai 2mo ago
  I meant more for their customers specifically
  - red-iron-pine 2mo ago
    we push customer code on mondays. it is monday. code is not being pushed. we do not bill. the outages they promised their customers are now different. this has costs for everyone downstream.
aurareturn 2mo ago
I was in the middle of a hot fix. Our pipeline goes through Github.
- parthdesai 2mo ago
  That's understandable, but surely there's a way to bypass it? You need to have a breakglass procedures lol
  - penultimatename 2mo ago
    “lol just use a workaround” doesn’t work in an environment with hundreds or thousands of employees coupled with audit, security, and other legal requirements to ship software.
    - parthdesai 2mo ago
      If you're actually bleeding money, you better believe you'll get permissions for a workaround, if you know what you're doing. It all ties back to the OP, where the issue you've might not be as bad as you think. I have been in situations where we have dropped all procedures to push a hot fix because we were actively bleeding money, and in situations where you know there is an issue, and you let it be.
      - penultimatename 2mo ago
        What an odd hill to die on. Maybe I just lost a $30M client because they wanted an issue fixed now. Maybe my company would get sued for violating their contractual obligations if someone went cowboy to bypass procedures. Maybe I pay for a service and I want that service to work consistently during core business hours.
    - bmitc 2mo ago
      "Just recreate GitHub internally."
      - red-iron-pine 2mo ago
        "just redo the whole CI/CD pipeline bro, it's easy"
phoronixrly 2mo ago
Imagine Github being critical infrastructure for you... The ineptitude...
- parthdesai 2mo ago
  We use github quite a bit as well. Github being down currently is not actively losing us money, i.e. having customer impact.

dewey 2mo ago

If you pay developers x money / day and one of their core tools is down for n hours during the day and they spend their money on HN instead that's pretty straight forward to calculate.

parthdesai 2mo ago
I meant more for their customers specifically
- red-iron-pine 2mo ago
  we push customer code on mondays. it is monday. code is not being pushed. we do not bill. the outages they promised their customers are now different. this has costs for everyone downstream.

aurareturn 2mo ago

I was in the middle of a hot fix. Our pipeline goes through Github.

parthdesai 2mo ago
That's understandable, but surely there's a way to bypass it? You need to have a breakglass procedures lol
- penultimatename 2mo ago
  “lol just use a workaround” doesn’t work in an environment with hundreds or thousands of employees coupled with audit, security, and other legal requirements to ship software.
  - parthdesai 2mo ago
    If you're actually bleeding money, you better believe you'll get permissions for a workaround, if you know what you're doing. It all ties back to the OP, where the issue you've might not be as bad as you think. I have been in situations where we have dropped all procedures to push a hot fix because we were actively bleeding money, and in situations where you know there is an issue, and you let it be.
    - penultimatename 2mo ago
      What an odd hill to die on. Maybe I just lost a $30M client because they wanted an issue fixed now. Maybe my company would get sued for violating their contractual obligations if someone went cowboy to bypass procedures. Maybe I pay for a service and I want that service to work consistently during core business hours.
  - bmitc 2mo ago
    "Just recreate GitHub internally."
    - red-iron-pine 2mo ago
      "just redo the whole CI/CD pipeline bro, it's easy"

phoronixrly 2mo ago

Imagine Github being critical infrastructure for you... The ineptitude...

parthdesai 2mo ago
We use github quite a bit as well. Github being down currently is not actively losing us money, i.e. having customer impact.