11 ms·
Incident with Issues and Webhooks – Resolved
- Group_B 5mo agoI'm moving everything to self hosted GitLab CE now. Will just use GitHub as a backup. This is so ridiculous at this point.
- Foorack 5mo agoGitLab vs Forgejo? Personally run GitLab but looking at moving, due to it's massive size.
- winddude 5mo agore there polymarket bets on if gitub will be up or down today yet?
- claudiusa 5mo agoworked for me pushing some commits about 20 minutes ago.
- Insanity 5mo ago_Usually_ the blast radius isn't "GH is down globally across all functionality". So it can work for you while still being either down for other regions, or at least degraded.
- bombcar 5mo agoPushing commits over SSH is often the most "reliable" thing, though you can get some fun situations where a commit is pushed and runners never ran, causing downstream FUN eventually.
- maccard 5mo agoI’ve seen this behaviour but IMO it’s a fatal design flaw of Actions. It shouldn’t be possible.
- tosti 5mo agoI don't do that anymore, but I can still access repositories and issues. Looks normal to me
- gritzko 5mo ago2027: "GitHub is up!"
- elevation 5mo ago"Three 8s of availability"
- baq 5mo agoI literally laughed out then shed a tear, because I'd actually take three 8s today.
- tardedmeme 5mo agoThree eights is more than a month of downtime every year. Today is the three eights.
- Philpax 5mo agoThis felt wrong to my intuition, but, no, you're right: (1-0.888) * 12 = 1.344.
- lelanthran 5mo ago2028: 3/8ths of availability
- Insanity 5mo agoAt this point, "GH is down" posts are competing with "Newest LLM Hype" for the HN front-page week over week. For my personal project, I've been considering moving everything over to Codeberg. Stability of GH being one reason, but I also like the idea of an alternative that is not strictly tied to a big tech company.
- SpyCoder77 5mo agoYour name summarizes all the GitHub uptime crap.
- eastbound 5mo agoAnd yet, you haven’t. That’s the problem with dominant platforms: Slight inconveniences + inertia are enough to ensure no-one moves (even without monopolistic abuse – and I’m talking about Microsoft here).
- htx80nerd 5mo ago"Claude Code is basically magic" spam hit the hardest. Temporarily side lined by GH status post. Maybe Claude Advertising is in a lull right now.
- fishgoesblub 5mo agoThis is genuinely pathetic. I wish I could do my job this poorly and still be employed.
- this_user 5mo agohttps://careers.microsoft.com/ https://careers.microsoft.com/
- cdud3 5mo agoIt's A Trap! ~ Admiral Ackbar
- tardedmeme 5mo agoKeep in mind that it's easy to call someone else incompetent whenever their thing doesn't work, but we have no full idea what's going on behind the scenes. They could be very competent people in an unwinnable situation (like being forced to use Azure).
- ecshafer 5mo agoAt this point we can just have a Bot repost this exact submission every day and it would be right more often than not.
- bigbuppo 5mo agoIt exists, but it runs as a github action, so it's not working right now.
- theworstname 5mo agoThat is beautiful
- tardedmeme 5mo agoWe couldn't, because HN prevents duplicate posts.
- indianhippie 5mo agoThis is reaching an unacceptable level of performance. There isn't a week that work isn't interrupted by GH.
- enraged_camel 5mo agoI would say we are way past unacceptable.
- lbourdages 5mo agoIt's been unacceptable for months, but now it's at the level of "we should actively look for alternatives".
- cenal 5mo agoAny centralized solution like GitHub is going to suffer the same fate as vibe coding chokes these services. The only option to have high uptime is to self host and most organizations can't do that easily. Time will tell if GitHub can scale up enough to meet demand.
- tardedmeme 5mo agoIt's a nice thought but I think the revealed preference from the history of the internet is that people actually only want centralised services, no matter what they say they want. People love to clown on the fediverse because of having to choose a server. Which is no different from email. I guess the difference is that their ISP used to give them email.
- ozgrakkurt 5mo agoNot really. It is possible to implement systems that handle a lot more scale than github has. This is proven by systems that exist today. It might be hard to create such systems using ruby and microslop AI management though
- petcat 5mo agoAI agents have changed the scalability properties of basically the entire internet. It used to be that GitHub could rely on a finite number of people interacting with their platform in real human ways in real observable patterns. So I'm assuming that they scale for those patterns, and optimize for the UI and UX hotspots. But now everyone's got a moltbot running 24/7, sometimes many, and it's completely overloading a lot of services. Especially services like GitHub which are very much agent-centric nowadays.
- Rzor 5mo agoSoon on your favorite prediction markets.
- int32_64 5mo agoAt least people are gaining knowledge of how to use the git remote command.
- matthew_hre 5mo agoHilariously, it looks like basically everything except Copilot is degraded. The jokes write themselves sometimes.
- cdrnsf 5mo agoCopilot's full functionality is only fractionally useful compared to what's currently degraded.
- Kwpolska 5mo agoCopilot is fully independent of the code forge parts of GitHub, so I would imagine it’s running on completely different infrastructure, without any hard dependencies on the Rails monolith.
- baderb 5mo agoI keep wondering what causing such a bad service from GH lately, is it the overuse of AI generated code? Are they trying to cut costs with the infrastructure?
- robertclaus 5mo agoThere was a post from Github a few weeks ago showing commit volume exploded from linear to exponential growth about 6 months ago. I don't know for sure, but I think they weren't ready for the scale out. Whether that means actual scaling issues or cost cutting because of the scale out, who knows.
- grepfru_it 5mo agoSomething something staff reductions something
- sailingparrot 5mo agoEveryone producing magnitude more code with AI agents. Numbers from GH COO here: https://x.com/kdaigle/status/2040164759836778878 https://x.com/kdaigle/status/2040164759836778878
- crote 5mo agoSo why aren't we seeing an explosion in feature shipping rate, or tech startups? If there is so much extra code, where is it going? Is everyone just creating giant piles of throwaway slop?
- Barrin92 5mo ago> Is everyone just creating giant piles of throwaway slop? yes, like the crypto or productivity 'industry', where every project exists to manage your other crypto and productivity projects, the dynamic of 'AI programming' seems to be to make software to manage your other two dozen AI things mysteriously enough, big open source projects that people actually use in the real world aren't seeing their issues closed at any higher rates, almost as if that requires actual maintenance and programming (which Fred Brooks told us this half a century ago)
- AlexB138 5mo agoGithub has published some incredible usage rate increase numbers, which they ascribe to the rise of agentic coding. At some point, they are going to have to change rate limits, cut free-tier usage, or find some other path to reducing load. It's clear that their infrastructure can't keep up with this significant increase, and it's unlikely that they're going to just absorb the increased costs themselves. Very curious to see what the future holds for Github.
- cdrnsf 5mo agoThey can't really cite the situation as a problem given their hand in creating and continuing it.
- petcat 5mo agoThe sysadmins didn't make any of those decisions.
- cdrnsf 5mo agoI suppose the idiocy of their parent company is their job security.
- nine_k 5mo agoIt's hard to talk about "them" as a singular entity. I bet that the "Copilot all the things!!11" faction mostly does not consist of GitHub SREs.
- Hamuko 5mo agoThe GitHub SREs are working for the Copilot company.
- cdud3 5mo agoSatya Nadella at the LlamaCon event in April 2025: "I’d say maybe 20%, 30% of the code that is inside of our repos today and some of our projects are probably all written by software." In particular Github, with it's copilot-next initiative, has probably so much AI generated code inside today that fixing all this new performance problems will need lots of human developer brains.
- SilverElfin 5mo agoIf this was a business running on its own, without Microsoft’s capital and anti competitive practices, they would have been out of business by now.
- dankobgd 5mo agooh no, anyway...
- Veyg 5mo agoHere we go again...
- ykhan11 5mo ago[dead]
- gejose 5mo agoGithub has 84.92% uptime in the last 90 days according to https://mrshu.github.io/github-statuses https://mrshu.github.io/github-statuses I don't know how this is even remotely close to acceptable.
- croes 5mo agoAt least one 9 … somewhere
- pluc 5mo agoIt isn't. Lots of unacceptable things going on these days and everyone seems to be accepting them just fine.
- tardedmeme 5mo agoWe should make an alternative git site, but how to acquire users?
- tantalor 5mo agoWhat do you need users for? GH is not a social network
- tardedmeme 5mo agoI don't know. Everyone seems to be using GitHub only because everyone else is using GitHub. Apparently that's important somehow. Me, I use "git init"
- 01HNNWZ0MV43FF 5mo agohttps://en.wikipedia.org/wiki/Network_effect https://en.wikipedia.org/wiki/Network_effect It's a lot easier to get bug reports and fixes when everyone is on the same auth system. That's why there is also a call for federated forges
- deleted 5mo ago[deleted]
- p33p 5mo agoOne of the most satisfying (and sad that it’s needed) GH CLI extensions I’ve (publicly) vibe coded has been https://github.com/Houstonwp/gh-down https://github.com/Houstonwp/gh-down Although I think I still ironically end up finding out about the outages on HN first.
- faangguyindia 5mo agoI have been writing lots of code lately And I still only have local git repository. I do not like why git doesn't have a built in bug or issue tracker and a kanban board or something. I wished Git and Fossil hybrid should have won. I thought about using fossil many times but it seems codex and claude have deeper integration with git. I don't like installing software which keeps growing into infinite feature Monster. Maybe I'll install gitea or forgjo idk. That's the last piece of puzzle remaining for me I've already mastered deployment and HA on bare metal from OVH and Hertzner, already have scaled to tons of users
- vaylian 5mo agoYou might be interested in radicle: https://radicle.dev/faq#how-does-radicle-handle-issues-pull-requests-code-reviews-etc-compared-to-github https://radicle.dev/faq#how-does-radicle-handle-issues-pull-... Radicle supports issues directly as part of the git object database.
- nine_k 5mo agoGitHub is actively used to do code review and bug tracking. There is a number of tools that offer it on top of git, in a distributed way, but it means that yo need to install them locally on every machine involved. What's worse, GitHub is widely used as a CI/CD solution, it runs massive amount of build pipelines and test suites. There is a ton of players in this space, too. GitHub's main value proposition was having all these things in one place, as a convenient web app, for free or for moderate money. So they're crushed by the success of their model.
- deleted 5mo ago[deleted]
- WolfeReader 5mo ago"I thought about using fossil many times but it seems codex and claude have deeper integration with git." Don't let "agentic" "coding" be the reason to avoid fossil. Fossil and other VCS are much easier for humans to use than Git is; there's no reason to have an LLM burning up tokens and the environment to do tasks you'd do yourself quickly and correctly.
- alfg 5mo agoWow, GitHub are no longer serious people. Unbelievable.
- Imustaskforhelp 5mo agoThe amount of damage Github has done to its brand is something so phenomenal that I think it might be studied in future case studies. Skype and Github were two microsoft products which everyone used until they degraded it. Windows itself can be included in it too now.
- chao- 5mo agoIt feels weird (sad?) that I'm starting to get a sixth sense for when Github is going to a service disruption. About an hour ago, clicking "Resolve Conversation" in a Pull Request failed a few times with an error message that appeared lower on the page (outside the viewport), and which I did not see the first few times. I had to reload the page after every few actions to get the server to register new ones. I told a colleague, and added "Github might be having an issue with some other service, and it's just bleeding over to PR comments? Maybe it will snowball into a larger outage?"
- throwaway613746 5mo ago[dead]
- romellem 5mo agoLiterally just had this same signal with PR review comments. Checked the status page, saw it was green, and (correctly) assumed “not for long!”
- thomas_viaelo 5mo ago[flagged]
- tailscaler2026 5mo ago> Co-authored by Co-pilot
- samuelknight 5mo agoI couldn't merge earlier but navigation around my project worked. Maybe if I generate even more PRs some will get through!
- deleted 5mo ago[deleted]
- saltyoldman 5mo agoIs Git decentralized?
- bigbuppo 5mo agoGit is, but with GitHub being treated as the golden repo for projects, it becomes the hub of other parts of development, like PRs and CI/CD pipelines.
- keybored 5mo agoYes. But in practice people need their `if any_tasks_failed: throw error` (“CI”) to be in the cloud and have three nines of uptime. Personally (but who am I) I don’t need to check whether my changes work on Windows 7 and SunOS in order to continue on a project most of the time.
- throwaway613746 5mo ago[dead]
- deleted 5mo ago[deleted]
- nine_k 5mo agoThe best time for GH to increase prices was 6 months ago, or so. No service is going to weather the storm of agentic code overload unscathed. But at least they could become an expensive-but-working solution instead of the sad comedy they're now, and thus keep their most lucrative customers.
- munificent 5mo agoI haven't seen Friendster-level stability like this in a long time.
- dickeeT 5mo agoreally need competitor for gh rn
- deathanatos 5mo agoGitlab, Forgejo, maybe Codeberg all exist. Vote with your wallet, if you hold the purse strings.
- PunchyHamster 5mo agoDoesn't matter if your dependencies are still using GH
- Zambyte 5mo agoThen use one. There are so many.
- tardedmeme 5mo agoit's git. Just do "git init" in some SSH accessible folder.
- fHr 5mo agoSelfhosted gitlab alpha versus the cloud Github beta
- m_w_ 5mo agoThis is really getting ridiculous - although people sometimes dismiss the "missing" status page because it includes copilot, it's worth noting that pull requests (95.5%) are even lower availability than copilot (96.4%). How am I expected to comment "LGTM" if I can't even get to the PR?
- hansmayer 5mo agoIs this finally the rise of the "multi-agentic os/platform" we've all been so eagerly awaiting?
- zkmon 5mo agoThe goals for code-versioning as they existed decades back, might be sliding into irrelevance now. Code is no longer a direct work artifact from humans, as it used to be. Back in the day, people wanted to persist their code and it's changes, because code was hard to write and test. People didn't use a versioning system for their compiled binaries though, because it was a machine output and can be created from source code. But there was no higher level "source" for generating the code itself. Now things have changed. We need to check what exactly is the human artifact that requires treasuring, persisting and versioning.
- hansmayer 5mo agoFlawed reasoning all around. Machine code and object code will look the same given the same source code, target platform, compilation and linkin params etc. How is AI-Slop-Code even close to that?
- pulkas 5mo agothey are testing the limits of ruby on rails. "Ultimately, if more companies treated the framework as an extension of the application, it would result in higher resilience and stability. Investment in Rails ensures your foundation will not crumble under the weight of your application. Treating it as an unimportant part of your application is a mistake and many, many leaders make this mistake." https://github.blog/engineering/architecture-optimization/building-github-with-ruby-and-rails/ https://github.blog/engineering/architecture-optimization/bu...
- mghackerlady 5mo agoIt'd be more useful to inform us when it is working at this point
- testemailfordg2 5mo agoProbably the solution is not using the centralized Github for all work related to agentic coding but rather a distributed/local github repository from get go. This way only what hits the centralized github and becomes public, is something vetted and signed off by the human in the loop, hopefully reducing the AI slop and increasing the quality of commits.
- mandeepj 5mo agoI think there should be another page 'GitHub is NOT down'. Needless to say, for all the other times it's Down.
- sergiotapia 5mo agoIf you have clowns bombarding the backend with agentic volume that is 200x the average, start charging them for usage. This is not sustainable and hurts actual customers.
- alansaber 5mo agoGithub vs cloudflare
- datadrivenangel 5mo agoEdit: the status page is still all green for today, but the incident is there. pre-edit: They just removed the incident and set the status page to all green as of 10:01 PT???
- DetroitThrow 5mo agoThey're trying to make it seem like it's not below 95% uptime really hard, so that might be the KPI they're trying to uphold.
- jfrbfbreudh 5mo agoReduce the free tier. I’ve made 4000 commits in the last 2.5 months. That’s just to main. And I push up tons of artifacts daily for regression testing. For $0.
- timmg 5mo agoHonestly, I hate "free tiers" for SAAS products like this. For about 5 minutes, Google had a pay-as-you-go service on GCP for git. I used to use that, because I wanted to own my stuff. But (I guess) because everyone used "free" github -- and like a lot of other Google services -- they sunsetted it. So now I'm using github for free. But would rather pay for my storage an usage with a (big) cloud provider.
- PunchyHamster 5mo agoThat's like 2 minutes of CPU usage for the repo part
- Kwpolska 5mo agoGitHub should add a slop tax. Co-authored by Claude? Pay up. Em-dashes in comments? Pay up. A lot of code written in a short window of time? Pay up.
- cphoover 5mo agoThey do that and open source projects that haven't left already will migrate.
- faangguyindia 5mo agoi just use local git repo and since i am a solo dev i push nowhhere. Though i plan to use my own server later.
- ethin 5mo agoHonestly, if GH keeps getting worse I may need to migrate away to Forgejo or something. The problem is GHA... Does anyone know of a service that is better and doesn't charge me an arm and a leg for runners (particularly MacOS ones)? I've been wanting to shift away from GHA for ages (because I hate it) but I don't know of any alternatives that are quite like it (or the costs involved).
- tommy_axle 5mo agoIf self-hosting the runners too then it's doable. Not 100% sure for Forgejo but with Gitea and act_runner it's possible and pretty economical if you have an extra mac mini.
- suryao 5mo agoFounder of WarpBuild here. We provide GHA runners that are much faster and about half the cost. This includes MacOS. Check us out. https://warpbuild.com https://warpbuild.com
- Bender 5mo agoWould it reduce their load if every AI instance both local and centralized had their own local git repo (and/or Gerrit instance) and then only committed for a sub-set of specific conditions were met?
- pbiggar 5mo agoCircleCI seems to not have the same problem: https://status.circleci.com/ https://status.circleci.com/
- josefritzishere 5mo agoMicroslop happens.
- pickleballcourt 5mo agoIs this trending in relation to Ghostty moving away from github?
- throwaway87281 5mo agoWhat's with Github Enterprise? At the workplace it's constantly broken as well.
- shevy-java 5mo agoMany of us predicted this. Microslop, 'xcuse me, Microsoft is killing off GitHub. Not necessarily by design or on purpose, but by slop-neglect effect. AI is eating the corporate brain that Microsoft used to have (well, a long time ago).
- deleted 5mo ago[deleted]
- zipy124 5mo agohttps://forgeperf.org/ https://forgeperf.org/ A useful website to show why GitHub is feeling the burn so much with higher usage!
- SoftTalker 5mo agoIt's almost as if some people don't understand that it's possible to develop software without using github. The people who warned about building dependencies on third party services are enjoying some schadenfreude now. Ironic that a source code control system that was designed to be distributed is now presenting problems due to the failure of centralized services.
- dbacar 5mo ago;) https://www.dayswithoutgithubincident.com/ https://www.dayswithoutgithubincident.com/ (from an above HN item)
- cedws 5mo agoAnyone have a nice way of mirroring between two forges such as GitHub and GitLab? I know git can push to multiple remotes but that only solves the problem for me, I still have to mirror commits from others. I want to keep repo configs in sync too.
- philipcristiano 5mo agoI’m using forgejo mirrors to push my local repo to GitHub, seems to work well. https://forgejo.org/docs/latest/user/repo-mirror/ https://forgejo.org/docs/latest/user/repo-mirror/
- ramon156 5mo agoLast time they had a very loose explanation that did not align with their timings. I wonder what Copilot is cooking up this time. Hardware upgrade?
- SpaceNoodled 5mo agoHow is this news? There's as low as a one-in-six chance it's down these days.
- yjaspar 5mo agoHow is Microsoft not able to scale? Is it an architecture limitation? That’s affecting trust big time…
- dlarsen5 5mo agoso funny that I was migrating my agent workflow off of github issues today and while I was trying to run a regression test against a run of the old github workflow it gave timeouts related to issues being down, been getting nothing but trouble the last few weeks from having github manage the project/issues
- shark1 5mo agoThe Internet infrastructure is not ready for all of these "generated" activity. Eventually it will be!