3 ms·

Disruption with Some GitHub Services

73 points by gpi 8mo ago

antiloper 8mo ago
Copilot has improved GitHub's uptime from five nines to nine fives, a four magnitude improvement!
MattIPv4 8mo ago
Again... Unicorns when trying to view files or PRs, errors trying to leave comments or review things if they do load.
rowanseymour 8mo ago
Ah the daily HN get together for devs waiting for Github to resolve its issues.
chilipepperhott 8mo ago
They're really struggling to keep their last 9 of uptime.
Maxious 8mo ago
They also had to remove GPT-5.3-Codex "as we focus on our platform reliability" https://x.com/github/status/2021040916451164412 https://x.com/github/status/2021040916451164412
xena 8mo ago
Another day, another downtime. Lovely.
roschdal 8mo ago
Who needs GitHub anyway?
- linhns 8mo ago
  Apparently, a lot of people on HN. Including me.
cpfleming 8mo ago
Nice to see you all again!
theappsecguy 8mo ago
I thought AI was supposed to replace Software engineers by now?
- f311a 8mo ago
  It's already, that's why Microsoft abandoned GitHub
paulbjensen 8mo ago
If you want to kill some (down)time, checkout https://www.mr-spankys-meatballs.com https://www.mr-spankys-meatballs.com - a FPS built with Svelte and ThreeJS.
hiccup_socks 8mo ago
i wonder what sorts of creative "haha copilot blah blah" and "haha microsoft blah blah" and "but they said AI would do xyz!" jokes we will get to see today. (please github, you need more uptime so that people can come up with some new jokes)
yabones 8mo ago
I have great compassion for the people who sort these things out. It takes a lot of skill and experience to pick up the pieces of something like Github and get it working properly again. I'm sure it's exhausting having problem after problem, endlessly chasing down sev-0 bugs in production. But I have nothing but disdain for the management that won't listen to those people.
- Xunjin 8mo ago
  It's time for some anonymous account or blog post try to share the insights on what the heck is happening, it's not normal, it can't be normal.
candiddevmike 8mo ago
For folks still on GitHub for $JOB, are you looking at alternatives yet?
- drcongo 8mo ago
  Yep. Daily. I can't bear GitLab though, so currently I'm thinking self-hosted Gitea or OneDev.
- andycowley 8mo ago
  That's only part of the problem really. The other thing is how many downstream things rely on GH being up. Things like Go packages, Helm charts, blah blah...
buildbot 8mo ago
“We believe we have found the cause of the problem and are working on mitigation.” Wow - they believe they’ve found it! Comforting. Having worked at Microsoft in the recent past I’m not surprised everything is starting to fall over a lot more. While there are tons of good people at Microsoft, the overall culture and number of people who stand in the way of getting stuff done well is immense. People view you or your team doing work as an existential threat to their team, and react accordingly. I don’t think the recent uptick in failures is related to vibe coding or AI and has everything to do with the corporate culture. Probably some poor product manager was tasked with closing 10k automated security flags/tickets and in doing so broke some load bearing microservice that depended on a kind of service principal that’s now not allowed.
leosanchez 8mo ago
DJ Khaled: Another One (Sorry for low effort comment).
drcongo 8mo ago
Moving GitHub to Azure has made GitHub as reliable as everything else that runs on Azure.
837263292029 8mo ago
As is tradition
yomismoaqui 8mo ago
If you can't deploy to production when some GitHub services hiccup maybe the problem is not on GitHub.

73 points by gpi 8mo ago

antiloper 8mo ago

Copilot has improved GitHub's uptime from five nines to nine fives, a four magnitude improvement!

MattIPv4 8mo ago

Again... Unicorns when trying to view files or PRs, errors trying to leave comments or review things if they do load.

rowanseymour 8mo ago

Ah the daily HN get together for devs waiting for Github to resolve its issues.

chilipepperhott 8mo ago

They're really struggling to keep their last 9 of uptime.

Maxious 8mo ago

They also had to remove GPT-5.3-Codex "as we focus on our platform reliability" https://x.com/github/status/2021040916451164412 https://x.com/github/status/2021040916451164412

xena 8mo ago

Another day, another downtime. Lovely.

roschdal 8mo ago

Who needs GitHub anyway?

linhns 8mo ago
Apparently, a lot of people on HN. Including me.

cpfleming 8mo ago

Nice to see you all again!

theappsecguy 8mo ago

I thought AI was supposed to replace Software engineers by now?

f311a 8mo ago
It's already, that's why Microsoft abandoned GitHub

paulbjensen 8mo ago

If you want to kill some (down)time, checkout https://www.mr-spankys-meatballs.com https://www.mr-spankys-meatballs.com - a FPS built with Svelte and ThreeJS.

hiccup_socks 8mo ago

i wonder what sorts of creative "haha copilot blah blah" and "haha microsoft blah blah" and "but they said AI would do xyz!" jokes we will get to see today. (please github, you need more uptime so that people can come up with some new jokes)

yabones 8mo ago

I have great compassion for the people who sort these things out. It takes a lot of skill and experience to pick up the pieces of something like Github and get it working properly again. I'm sure it's exhausting having problem after problem, endlessly chasing down sev-0 bugs in production. But I have nothing but disdain for the management that won't listen to those people.

Xunjin 8mo ago
It's time for some anonymous account or blog post try to share the insights on what the heck is happening, it's not normal, it can't be normal.

candiddevmike 8mo ago

For folks still on GitHub for $JOB, are you looking at alternatives yet?

drcongo 8mo ago
Yep. Daily. I can't bear GitLab though, so currently I'm thinking self-hosted Gitea or OneDev.
andycowley 8mo ago
That's only part of the problem really. The other thing is how many downstream things rely on GH being up. Things like Go packages, Helm charts, blah blah...

buildbot 8mo ago

“We believe we have found the cause of the problem and are working on mitigation.” Wow - they believe they’ve found it! Comforting. Having worked at Microsoft in the recent past I’m not surprised everything is starting to fall over a lot more. While there are tons of good people at Microsoft, the overall culture and number of people who stand in the way of getting stuff done well is immense. People view you or your team doing work as an existential threat to their team, and react accordingly. I don’t think the recent uptick in failures is related to vibe coding or AI and has everything to do with the corporate culture. Probably some poor product manager was tasked with closing 10k automated security flags/tickets and in doing so broke some load bearing microservice that depended on a kind of service principal that’s now not allowed.

leosanchez 8mo ago

DJ Khaled: Another One (Sorry for low effort comment).

drcongo 8mo ago

Moving GitHub to Azure has made GitHub as reliable as everything else that runs on Azure.

837263292029 8mo ago

As is tradition

yomismoaqui 8mo ago

If you can't deploy to production when some GitHub services hiccup maybe the problem is not on GitHub.