7 ms·
Incident with Pull Requests, Issues, Git Operations and API Requests
- trenchgun 4mo agoBut everything else still works fine, right?
- maxnoe 4mo agoGitHub Incident again/
- denysvitali 4mo agoAt this point we can even stop specifying that it's GitHub...
- sibidharan 4mo agoAre they running paid marketing campaigns for Gitlab ?
- EduardoBautista 4mo agoI'd consider self-hosting GitHub Enterprise before putting my team through the pain of Gitlab.
- ricardbejarano 4mo agoHow is it painful to use GitLab? Curious, as a user of both, I find them both nice. I like GitLab CI/CD more than I do GHA, but that's personal preference/bias more than anything objective.
- tux3 4mo agoGitlab CI has some tech debt from accumulating geological layers of different ways to do things, but overall it's pretty good, it scales to more complicated setups, and it's not too painful. Now the best way to use GHA is to do the bare minimum. Put all your CI logic in a script that you can test locally, and just have GHA run your script. Even that is painful. And, somehow, impossible to make secure without having spent 5,000 hours reading all the previous ways people got pwn'd by Github Action's horrendous security model. My main problem with Gitlab is that after years I still can't find what I'm looking for in the UI. It's always exactly in the third place I look. Otherwise Gitlab has been good. Even self-hosted works pretty well.
- dijksterhuis 4mo agoexpanding on the parent a little * GiLab — Ops centric * GitHub — Developer centric if you just want somewhere to stick a code repo and build a release every so often — dont use gitlab, you will not enjoy it. > My main problem with Gitlab is that after years I still can't find what I'm looking for in the UI. i still get lost too after several years daily driving gitlab. this is the Ops centric thing. they provide a lot of options. lots of options is good for Ops. > Now the best way to use GHA is to do the bare minimum yeah, i’m an ops guy, so the maintaining custom actions stuff on github is horrible for me vs click a button and move on with my day — once i find the button that is! xD
- 000ooo000 4mo ago>GitHub — Developer centric I'm looking at the branch listing and I can't see a single hash. I dunno who GitHub is for but it isn't devs.
- EduardoBautista 4mo agoEverything about their UI/UX screams of doing the bare minimum to check off a box on a feature list. It reminds me of Jira.
- nucleardog 4mo agoAs a daily GitLab user, I'd say that would be the main criticism I could levy at it as well. It does feel like there are a number of "and the kitchen sink" type features that are just there to check a box in a RFP or something. That said, are the majority of people actually even _using_ those features? For us we're essentially just using GitLab for git, merge requests, and CI pipelines. A couple places we use the static page hosting. (First thing I do whenever I create a new repository is go into the settings and just uncheck _all_ the boxes.) All of that core functionality works really well and is more than polished enough from my point of view.
- IshKebab 4mo agoGitlab is pretty decent. Honestly I would say there's not much between GitHub and Gitlab. Gitlab's CI is more powerful than GitHub's IMO, but the UX is a bit worse. But it's really marginal. They're both slow and have tons of long-standing missing features. But they're ok. I'd definitely rather use Gitea/Gogs/Forgejo, and maybe Tangled if it supported normal setups (e.g. private repos).
- ramon156 4mo agoSelf-hosting forgejo under tailscale + mirroring public repos through GitHub Has worked wonders for me :)
- varun_ch 4mo agoForgejo is fantastic. I do think it could use a fresh coat of paint from a designer but it’s otherwise really good. Gitea (what Forgejo forked from) recently stole the sidebar on repos from GitHub and I think that would be great for Forgejo to steal too… Forgejo themed by Codeberg: https://codeberg.org/forgejo/forgejo https://codeberg.org/forgejo/forgejo (the codeberg theme is extremely low contrast) Forgejo default: https://v15.next.forgejo.org/pparaxan/quark https://v15.next.forgejo.org/pparaxan/quark Forgejo themed by Lix: https://git.lix.systems/lix-project/lix https://git.lix.systems/lix-project/lix Gitea: https://gitea.com/gitea/awesome-gitea https://gitea.com/gitea/awesome-gitea Gitea themed by Blender: https://projects.blender.org/blender/blender https://projects.blender.org/blender/blender I personally like Blender’s Gitea theme better than the rest but I guess that’s subjective. In dark mode I do not like the low contrast Codeberg theme or the default Forgejo theme, but all of the instances custom themes look great. As far as Git forges go in general though.. tangled is very pretty https://tangled.org/tangled.org/core https://tangled.org/tangled.org/core I think more power user oriented software should be comfortable with compact interfaces
- maxfurman 4mo agoThat Blender Gitea theme is really nice! I wonder why exactly it's so much easier on the eyes? In a lot of ways all of these are "just Github" with minor changes, so the one that is actually better really stands out.
- KptMarchewa 4mo agoIt's so unfortunate that Gitlab is a complete mess, that GitHub has no real competition now. I can only think that few months to a year from now there will be _something_ that works on an enterprise scale.
- xnorswap 4mo agoHave you forgotten about Azure Dev Ops aka Visual Studio Team System aka Team Foundation Server*? Yes, it's still Microsoft, but they've forgotten about it, so it runs entirely adequately and is actually a surprisingly okay github replacement. It does nothing special, but it does do everything, just in a way you often would rather it wouldn't. It doesn't have the flexibility of JIRA for the ticketing, and the deployment machinery doesn't have the fanciness ( and vendor threat ) of chaining github actions, but it does handle both. I haven't used gitlab, so I'm curious to hear what makes it a "complete mess" too. * Microsoft's headless chicken naming strategy in full force, it's a miracle they haven't yet renamed and rebranded it to align with copilot yet.
- zdragnar 4mo agoOne does not mention TFS in polite company
- manytimesaway 4mo agoDon't forget CodePlex!
- KptMarchewa 4mo agoOnly alternative outside of GitHub and GitLab I've used was Bitbucket, and it was worse - but this was time when GitHub was good.
- kevin_thibedeau 4mo agoThere are plenty of open source forges with a better UX.
- cedws 4mo agoI'm so done with GitHub.
- gen220 4mo agohttps://isgithubcooked.com https://isgithubcooked.com Normally I defend GH in the comments of these incidents but it’s been an impressively bad month by their standards, even when you filter for critical components filter out sev-2’s and 3’s.
- btown 4mo agoIs the “streak” days of continuous uptime, or of days with at least one downtime incident? I think it’s the latter :]
- joshuaissac 4mo agoIt looks like it is the number of consecutive days with no incident. If you look at 31 Dec 2025, that corresponds to an 8-day period with no incidents.
- isityettime 4mo agoI guess that also means this year GitHub has not yet made it a single week without an outage of some kind.
- gen220 4mo agoIt's a streak for continuous uptime, and yeah it is fairly depressing to imagine overseeing that :/
- EduardoBautista 4mo agoMay has been filled with critical issues. It seems it's getting worse over time.
- hbn 4mo agoCommits are up 14x year-over-year https://x.com/kdaigle/status/2040164759836778878 https://x.com/kdaigle/status/2040164759836778878
- Arbortheus 4mo agoFed up and bored of this
- rvz 4mo agoAgain? It was just yesterday [0] that GA was down and another incident today? I am convinced that Copilot and Tay.ai are destroying GitHub and there is no CEO of GitHub to contact. Now will you please self-host as I said 6 years ago? [1] [0] https://news.ycombinator.com/item?id=48278635 https://news.ycombinator.com/item?id=48278635 [1] https://news.ycombinator.com/item?id=22867803 https://news.ycombinator.com/item?id=22867803
- rob 4mo agoYou think a Microsoft chatbot from 2016 is destroying GitHub?
- rvz 4mo agoAt this point, you might as well say that is what's happening at GitHub with the help of GitHub Copilot since nothing has changed and has only gotten worse over time.
- insider123 4mo agoMicrosoft corporate culture destroys Github reputation and tech.
- KptMarchewa 4mo agoGitHub famously does not have a single 9 of uptime.
- looperhacks 4mo agoMaybe we should start posting av story when GitHub has been fine for some time instead of posting every incident
- voidUpdate 4mo agoThey may have gotten down to only 2 nines on most of their services, but at least the LLM is still running at full power! must increase value for shareholders
- dzonga 4mo agogit is supposed to be decentralized. maybe it's time to revert back to the central idea of git & not centralize around a particular provider. for issues - mailing list will do. you can always slap a beautiful ui if you want to or a tui (as is the fad) these days. actions can also be decentralized via an API spec & webhooks.
- xnorswap 4mo agoBefore clicking, I assumed this was going to be a write-up of the one from a few days ago instead of an entirely new incident.
- jamdav16 4mo agoI assumed it was the one from yesterday! Silly me.
- spaceman_2020 4mo agois it me or ever since AI coding became the norm, there have been way more outages with otherwise reliable services? I get downtime on Supabase every few weeks. Even Cloudflare. And now Github
- julianlam 4mo agoNot just you, but uncertain whether it's due to unreviewed slop going to production, or increased demand due to slop generation.
- hansmayer 4mo agoNo, it's not just you. It is fairly obvious what's happening - the same old Entshittificators now have a great tool to up the speed of entshitification by 100x - thus these crappy outages every other day.
- throwatdem12311 4mo ago> is it me or No, of course not.
- sedimannapoleon 4mo ago[dead]
- chris_money202 4mo agoYes, because that caused the usage of the services to skyrocket, GitHub runs on Azure and Azure is experiencing capacity strain due to AI, so GitHub's services are struggling to auto-scale
- voncheese 4mo agoPer a report that came out the other day, the GitHub move to Azure has been slowed down (i.e. I don't think it's done). But maybe you have newer/better info than me
- skywhopper 4mo ago
- fen4o 4mo agoTried to do a git push - it succeeded after 3 mins. Then I wanted to open a PR and it failed with a 500 error. Facepalmed and decided that this is it for today.
- Hamuko 4mo agoYesterday my CI runs wouldn't even be created because Actions was eating shit, and today my CI runs get created but fail because the API is eating shit. Fun.
- ckorhonen 4mo agoThis is getting ridiculous. One particularly concerning thing I’m seeing is that pull requests on both the web UI and API aren’t reflecting all commits or branch changes consistently. It would be very easy to merge something without realizing you’re not actually reviewing the full diff.
- mikeocool 4mo agoYeah, I've had several occasions recently (seemingly not related to any incidents on the status page) where I've had to wait 20 minutes to an hour to be able to open a PR, because Github didn't recognize my branch had any new commits compared to the base branch.
- dude250711 4mo agoA taste of the agenticaly-developed world.
- refulgentis 4mo ago[flagged]
- drcongo 4mo agoFor years we had a GitHub status thing in our Slack but I had to remove it about a year ago because the noise got too much, it would be unbearable in 2026.
- dist-epoch 4mo agoGitHub is not agent scale. Multiple companies are trying to create new versioning primitives/architectures which can handle machine-level code generation - 1 commit per second per repo. It's like switching from horse buggies to automobiles, the whole worlds needs re-architecturing to handle the new load. The age of boutique hand-coding is being replaced by the age of industrial software factories.
- julianlam 4mo agoWhy the heck would you want to do this. Using git as your undo chain sounds like a pretty awful thing to do.
- dpkirchner 4mo agoI think it'd be pretty neat to be able to rebase my undo history on to a remote branch someone else is working on.
- fapjacks 4mo agoThis is smart but also hilarious.
- andyjohnson0 4mo ago> GitHub is not agent scale. Is the scaling issue with git or github?
- skinfaxi 4mo agoThis seems odd to me. Why would you need to commit every second?
- gedy 4mo agoAnd push to remote as well? Seems not thought out
- swiftcoder 4mo ago> new versioning primitives/architectures which can handle machine-level code generation - 1 commit per second per repo. This is not a particularly novel level of scale. Facebook's mercurial backend had to handle >5,000 developers committing to the singular monorepo long before LLMs were a thing
- hansmayer 4mo agoIt seems before AI eats software, its going to first eat GH and Microsoft.
- throwatdem12311 4mo agothey didn’t think the leopard would eat their face!
- hansmayer 4mo ago[flagged]
- tomhow 4mo agoWe've banned this account for repeatedly posting abusive comments. We've asked you before to observe the guidelines, and the recent trend is getting much worse. When this happens we eventually have to assume you want to be banned. If you don't want to be banned, you can email us (hn@ycombinator.com) and commit to using HN as intended.
- abhashanand1501 4mo agoas a github user, we are paying for the slow git operations through our github action minutes, if someone from GH is here, will you be compensating for it?
- throwatdem12311 4mo agoMaybe GitHub needs to freeze free repository creation until they get this under control because this is ridiculous.
- embedding-shape 4mo agoI mean, if we're talking about "fixing" the symptoms of the downtimes rather than the sources and causes, I guess they could just null route github.com until they have things under control? Personally, I think they'd have more luck actually attacking the source, what that might be. Somehow I think Microsoft's push for "Every developer only use AI for development, no manual thinking/coding from now on" is the detrimental step, seemingly many companies are still discovering the right approach. Put a freeze to that, and I'm fairly sure you'd see less downtime pretty much immediately, unless all real engineers already left the company, I'm sure I would have at this point.
- throwatdem12311 4mo agoIt’s not just their own slop that’s causing this, it’s also caused by the tsunami of slop being uploaded by vibe coders. If you want to upload to GitHub, you should pay. The days of charitably giving away compute for the “open source communities” are over. Grandfather existing public repositories in, then cut it off. Stop the bleeding. It doesn’t have to be forever.
- goda90 4mo agoOr maybe they need to bring back quality assurance expertise to the company.
- throwatdem12311 4mo agoYes that would be part of getting things under control, of course.
- ethagnawl 4mo agoAnd/or move more contextually aware humans with 10K+ hours of hard won experience and fear of failure/sense of pride back into the loop.
- renehsz 4mo agohttps://GiveUpGithub.org https://GiveUpGithub.org
- chrisweekly 4mo agoGood link - why and how to ditch GitHub.
- emartinez-dev 4mo agoI thought it was the yesterday's thread but no, here we go again
- rozab 4mo agoIf you go to www.githubstatus.com, the downtime is not showing in the chart. I was annoyed enough yesterday when I visited this page to figure out why my Actions had failed and was greeted with big green ticks and only a tiny red rectangle halfway down the page to indicate the problem. This time they've just scrubbed the evidence outright?
- madeofpalk 4mo agoIt was previously showing, but I believe the incident has bee resolved now. At least, PRs work for me when they previously didn't.
- Symbiote 4mo agoThe "Git Operations" chart is showing all green, but several of the recent blocks have a note showing there was an outage. Today's is green, even though there was an outage.
- eithed 4mo agoI'd appreciate if they'd not mark the incident as resolved when there's still fallout - ie: my commits didn't display on the branch, my actions didn't run It's the same issue as the other day - display message at the top admitting that cache needs to be refreshed (or whatever the wording was)
- Systemic33 4mo agoSomeone linked this third-party "honest" status page: https://mrshu.github.io/github-statuses/ https://mrshu.github.io/github-statuses/ Seems more accurate with my experience of GitHub.
- robin_reala 4mo agoGood that the Billing functionality is still at 3 nines at least.
- hydrogenbon007 4mo agoThe software reliability and uptime is going bad across the industry, railway, github etc wild that there is a large pattern forming up of unreliable software being pushed
- shevy-java 4mo agoI think it is time to decouple GitHub from Microsoft. Microsoft has shown irresponsible behaviour - and this continues. They keep on going at it until nothing works anymore. Typical microslop work.
- pocksuppet 4mo agoYou don't get to control that. It is Microsoft's right to do whatever it wants with GitHub - it could shut down tomorrow, or demand face ID. If you want to control what happens with a thing, you have to make the thing instead of letting someone else make the thing and sell it to Microsoft. Your choice is to accept the product that exists on the market, switch to another product that exists on the market (such as Codeberg or self-hosted Forgejo), make your own product, or not use any.
- OkayPhysicist 4mo agoYou missed the option to "lobby the government to tell Microsoft to straighten up and fly right". Microsoft is a corporation, a legal entity only created with the permission and grace of the State of Washington.
- pocksuppet 4mo agoSure, I guess skywriting prayers to the spaghetti monster is also an option, but neither of those will do anything.
- deleted 4mo ago[deleted]
- deleted 4mo ago[deleted]
- gred 4mo agoNew PR: revert GitHub software and infrastructure to version of June 1st, 2018. New PR: disable new user signups for 6 months HR initiative: all future KPIs automatically require three-nines availability; all bonuses are forfeited, regardless of accomplishments, if annual availability falls below target HR initiative: fire CEO and CTO
- rahkiin 4mo agoNew PR: disable Github API New PR: block (ai) bots through attestation to make usage predictable
- thr0w4w4y1337 4mo agoGithub does not have a CEO
- gred 4mo agoWe're halfway there!
- 0xblinq 4mo agoFinance initiative: Undo the Microsoft purchase
- fapjacks 4mo agoImagine if we did this for countries.
- hehe1 4mo agodthtrj
- __vivek 4mo agohttps://mrshu.github.io/github-statuses/ https://mrshu.github.io/github-statuses/
- Robdel12 4mo agoI know it’s super fun to shit on GitHub and everyone’s favorite thing to say is “build a competitor!” They’re trying to scale from 1 billion commits last year to over 14 billion this year. I have zero desire to try and manage that scaling. Basically being DDOS’d by agents all day now. https://x.com/kdaigle/status/2040164759836778878 https://x.com/kdaigle/status/2040164759836778878
- sethops1 4mo agoIf it were humans using the site it might be super motivating, but knowing that it's 99% bots producing AI slop, yeah I'd be looking for a new job.
- fg137 4mo ago> I have zero desire to try and manage that scaling. Last time I checked, this is called "doing their job". You are free to choose what company you want to work for, but for github, if they are offering the services, often as paid services, they have an obligation to keep things running which is called SLA. Otherwise people will leave (and are leaving)
- OsrsNeedsf2P 4mo agoGithub Pull Requests are down to 1 nine of availability
- ale 4mo agoI think it’s about time to backup all of my GH repos.
- markus_zhang 4mo agoThe moral of the story is: we don’t even need one 9. OK now people can layoff even more engineers and feed the tasks to AI. /s
- carreau 4mo agoReminder that after more than a month, many repos _still_ don't show all PRs and Issues https://github.com/orgs/community/discussions/193463 https://github.com/orgs/community/discussions/193463
- rldjbpin 4mo agoamong all the recent incidents, this was probably the first time i experienced any service disruption. even comparing branches were giving 500 in the web ui. even github copilot in their new session limits now also mention if github is going through an incident, which is plainly comical. fun times!
- My_Name 4mo agoI'm finding this amusing because today, I am in the process of removing all my code from GitHub and cloning it to a local Git that has a webhook to a local releases server.
- diogogberger 4mo ago[flagged]
- killamdiaz 4mo ago[flagged]