4 ms·
> Github’s servers are constantly on fire as their usage increased something like 50x due to LLM sloppers pushing large amounts of trash code I sympathized wit
by jjice 2mo ago
> Github’s servers are constantly on fire as their usage increased something like 50x due to LLM sloppers pushing large amounts of trash code
I sympathized with them when they said this a handful of months ago, but then I saw this [0] page that shows how it's been shot for years prior (which tracks with my memory).
I feel for the GH engineers that have to deal with this, especially the SREs. I also don't hate the downtime right now, as I'll make a cup of coffee and do something else. I will say though, I did have a hotfix a week or so ago during the Actions outage, which really was a pain.
You're right that getting angry and ranting isn't the right reaction here, but I do no give them the LLM load excuse. I don't give them an out for having awful uptime during work hours for a product we pay quite a bit for as an org.
[0] https://damrnelson.github.io/github-historical-uptime/ https://damrnelson.github.io/github-historical-uptime/
- Topfi 2mo agoGithub being under the CoreAI division probably also doesn't help the engineers prioritize addressing infrastructure issues and makes using LLM load as an excuse feel self-inflicted. Akin to feeling sorry when a pyromaniacs house burns down...
- debarshri 2mo agoIt looks like right after Microsoft acquisition this started going off. This is the outcome of violating single responsibility principle in business.
- twistedpair 2mo agoThey _are_ the LLM coding agent vendor, and are _owned_ by MS, the ~majority~ biggest shareholder of OpenAI. How can you NOT consider that 20x+ scaling in your capacity roadmap projections, where you are trying to get everyone to use these agents as part of your core OKRs? Or, maybe your #1 IT priority was moving everything to Azure instead ;)
- rafram 2mo ago> the majority owner of OpenAI Largest shareholder (27%), not majority.
- chilipepperhott 2mo agoPlurality should have been the operative word.
- chrisjj 2mo agoThey've had a couple of months and still have not brought in sufficient extra capacity?? What's their excuse?