5 ms·

Could be. But 99% of the repos are static garbage with no PR nor actions. They mentioned they have some elasticsearch reindexing going to, I would guess they n

by motbus3 6mo ago

Could be. But 99% of the repos are static garbage with no PR nor actions. They mentioned they have some elasticsearch reindexing going to, I would guess they needed to regard or move stuff and something didn't work well. But if I understood it right they mentioned the PRs ES index which they didn't shared proof increased as the number of repos. It might be anything. It seems they lost huge chunks due to layoffs and structural changes and MS which has the reverse golden Midas touch. This is just pure speculation but also now there is no reason for MS to keep GH working. They absorbed all code they wanted. Now they can let it burn. Would be even better for them if that happened

parthdesai 6mo ago
Serious question, have you been part of an org that had to scale orders of magnitude very quickly? Anyone who has been part of that journey knows how painful it really is. A lot of times the systems to fail at all levels, and you have to redesign it from the first principles.
- owebmaster 6mo ago
  > you have to redesign it from the first principles And that start by layoffing your best engineers, I guess
- HWR_14 6mo ago
  Is GitHub scaling by orders of magnitude though? That would be an insane increase at this stage of their lifecycle.
  - ambicapter 6mo ago
    If they're suffering the onslaught of ai slop, it's possible.
  - jodrellblank 6mo ago
    They say it is at least one order of magnitude[1]; "our plan to increase GitHub’s capacity by 10X in October 2025 .. By February 2026, it was clear that we needed to design for a future that requires 30X today’s scale." [1] https://github.blog/news-insights/company-news/an-update-on-github-availability/ https://github.blog/news-insights/company-news/an-update-on-...
    - ori_b 6mo ago
      Note the lack of concrete numbers on how much they have scaled. Somebody may have just asked an LLM for projections.
      - Barbing 6mo ago
        Would Microsoft lawyers OK that? GitHub would have obligations to MS investors to make accurate projections just like Microsoft itself, right?
        HWR_14 6mo ago
        I don't think it's an issue here. If the investor relations people put it out, it would be. But in this case it is closer to marketing.
      - codechicago277 6mo ago
        https://gitcharts.com https://gitcharts.com shows ~310 million public repos today, vs. 250 million in April 2025 (according to the wayback machine). Large increase, but nothing existential.
  - nicoburns 6mo ago
    I wouldn't be surprised. Have you not noticed the sheer volume of slop being posted everywhere these days? Almost all of that is hosted on Github. And some of those repos have insane commit frequencies.
- dijit 6mo ago
  > Serious question, have you been part of an org that had to scale orders of magnitude very quickly? I have, but it depends what you mean. Scenario 1: e-commerce SaaS (think: Amazon but whitelabel, and before CPUs even had AES instructions); Christmas was "fun". Scenario 2: Video Games. The first day is the worst day when it comes to scale. Everything has to be flawless from day 0 and you get no warning as to what can go wrong. Yet, somehow, I managed to make highly reliable systems. In scenario 1; I had an existing system that had to scale up and down with load, this was before there was cloud and hardware had a 3-4 month lead time, so most of the effort was around optimising existing code, increasing job timeouts and "quenching" sources that were expensive. We used to also do so 'magic' when it came to serving requests that had session token or shopping cart cookie. In scenario 2; we have a clean-room implementation and no legacy, which is a blessing but also a curse, there's no possibility to sample real usage: but you also don't need to worry about making breaking changes that are for the better. With legacy you have to figure out how to migrate to the new behaviour gradually. So, pro's and con's... but it's not like handling huge load hasn't been done before, computers are faster than they ever have been and while my personal opinion is that operational knowledge is dying (due to general distain for people who actually used to run systems that scale: not just write hopeful "eventually consistent" yaml that they call deterministic) - the systems that do exist today hold your hand much better than they did for me 20 years ago. And I ran 1% of web traffic with an ops team of 5 back then. So, idk what's going on here. EDIT: Likely people are flagging me because I sound arrogant (or I hurt their feelings by talking bad about YAML-ops), but all I am doing is answering the question presented based on my experience.
  - Dwedit 6mo ago
    I think you meant "green fields" and not "clean room"? Clean room refers to reverse engineering an existing program to create specifications, then having another team implement the specifications without legal risk from involving the original.
    - dijit 6mo ago
      Yes I did, sorry! You are right. :)
jonfw 6mo ago
> Could be. But 99% of the repos are static garbage with no PR nor actions. But the 1% of repos that do have PRs and actions are likely going to be seeing enormous increases in volumes I have been a part of two very large companies with self hosted gits and I've seen enough to be confident that this is an incredibly hard thing to manage
- fourseventy 6mo ago
  Ya but they are owned by freaking microsoft and have billions of dollars and employees to throw at the problem. The outage problems shouldn't be happening period.
  - jonfw 6mo ago
    Easy to say that! Some problems are legitimately hard to solve though. Github is likely seeing usage patterns that have never been seen before and I bet some of these failure modes are novel If you are at the limits of your architecture you may need to re-write things, and if you are rewriting things you can not arbitrarily speed that up by throwing dollars at it.
    - motbus3 6mo ago
      It is not like MS is involved with AI and say they can make anything in minutes with AI too
    - joepie91_ 6mo ago
      That's entirely a predicament of Microsoft's own making, though. Don't forget that they're the ones who launched "AI" programming into the hype cycle to begin with. So it's entirely reasonable to hold them as a company responsible for the resulting outages, which indeed shouldn't be happening. Dogs, fleas, and so on.
giancarlostoro 6mo ago
At that point, make it lazy indexing? Who cares that I can't find a repo that was made 10 seconds ago, or even 15 minutes ago? No seriously, who cares? Search to that level of nuance is not mission critical, I don't care what anyone says, you'll live if you wait another 15 minutes or even an hour. Their search has been terrible since their last major set of search changes where they overhauled it completely either way.

by motbus3 6mo ago

parthdesai 6mo ago

Serious question, have you been part of an org that had to scale orders of magnitude very quickly? Anyone who has been part of that journey knows how painful it really is. A lot of times the systems to fail at all levels, and you have to redesign it from the first principles.

owebmaster 6mo ago
> you have to redesign it from the first principles And that start by layoffing your best engineers, I guess
HWR_14 6mo ago
Is GitHub scaling by orders of magnitude though? That would be an insane increase at this stage of their lifecycle.
- ambicapter 6mo ago
  If they're suffering the onslaught of ai slop, it's possible.
- jodrellblank 6mo ago
  They say it is at least one order of magnitude[1]; "our plan to increase GitHub’s capacity by 10X in October 2025 .. By February 2026, it was clear that we needed to design for a future that requires 30X today’s scale." [1] https://github.blog/news-insights/company-news/an-update-on-github-availability/ https://github.blog/news-insights/company-news/an-update-on-...
  - ori_b 6mo ago
    Note the lack of concrete numbers on how much they have scaled. Somebody may have just asked an LLM for projections.
    - Barbing 6mo ago
      Would Microsoft lawyers OK that? GitHub would have obligations to MS investors to make accurate projections just like Microsoft itself, right?
      - HWR_14 6mo ago
        I don't think it's an issue here. If the investor relations people put it out, it would be. But in this case it is closer to marketing.
    - codechicago277 6mo ago
      https://gitcharts.com https://gitcharts.com shows ~310 million public repos today, vs. 250 million in April 2025 (according to the wayback machine). Large increase, but nothing existential.
- nicoburns 6mo ago
  I wouldn't be surprised. Have you not noticed the sheer volume of slop being posted everywhere these days? Almost all of that is hosted on Github. And some of those repos have insane commit frequencies.
dijit 6mo ago
> Serious question, have you been part of an org that had to scale orders of magnitude very quickly? I have, but it depends what you mean. Scenario 1: e-commerce SaaS (think: Amazon but whitelabel, and before CPUs even had AES instructions); Christmas was "fun". Scenario 2: Video Games. The first day is the worst day when it comes to scale. Everything has to be flawless from day 0 and you get no warning as to what can go wrong. Yet, somehow, I managed to make highly reliable systems. In scenario 1; I had an existing system that had to scale up and down with load, this was before there was cloud and hardware had a 3-4 month lead time, so most of the effort was around optimising existing code, increasing job timeouts and "quenching" sources that were expensive. We used to also do so 'magic' when it came to serving requests that had session token or shopping cart cookie. In scenario 2; we have a clean-room implementation and no legacy, which is a blessing but also a curse, there's no possibility to sample real usage: but you also don't need to worry about making breaking changes that are for the better. With legacy you have to figure out how to migrate to the new behaviour gradually. So, pro's and con's... but it's not like handling huge load hasn't been done before, computers are faster than they ever have been and while my personal opinion is that operational knowledge is dying (due to general distain for people who actually used to run systems that scale: not just write hopeful "eventually consistent" yaml that they call deterministic) - the systems that do exist today hold your hand much better than they did for me 20 years ago. And I ran 1% of web traffic with an ops team of 5 back then. So, idk what's going on here. EDIT: Likely people are flagging me because I sound arrogant (or I hurt their feelings by talking bad about YAML-ops), but all I am doing is answering the question presented based on my experience.
- Dwedit 6mo ago
  I think you meant "green fields" and not "clean room"? Clean room refers to reverse engineering an existing program to create specifications, then having another team implement the specifications without legal risk from involving the original.
  - dijit 6mo ago
    Yes I did, sorry! You are right. :)

jonfw 6mo ago

> Could be. But 99% of the repos are static garbage with no PR nor actions. But the 1% of repos that do have PRs and actions are likely going to be seeing enormous increases in volumes I have been a part of two very large companies with self hosted gits and I've seen enough to be confident that this is an incredibly hard thing to manage

fourseventy 6mo ago
Ya but they are owned by freaking microsoft and have billions of dollars and employees to throw at the problem. The outage problems shouldn't be happening period.
- jonfw 6mo ago
  Easy to say that! Some problems are legitimately hard to solve though. Github is likely seeing usage patterns that have never been seen before and I bet some of these failure modes are novel If you are at the limits of your architecture you may need to re-write things, and if you are rewriting things you can not arbitrarily speed that up by throwing dollars at it.
  - motbus3 6mo ago
    It is not like MS is involved with AI and say they can make anything in minutes with AI too
  - joepie91_ 6mo ago
    That's entirely a predicament of Microsoft's own making, though. Don't forget that they're the ones who launched "AI" programming into the hype cycle to begin with. So it's entirely reasonable to hold them as a company responsible for the resulting outages, which indeed shouldn't be happening. Dogs, fleas, and so on.

giancarlostoro 6mo ago

At that point, make it lazy indexing? Who cares that I can't find a repo that was made 10 seconds ago, or even 15 minutes ago? No seriously, who cares? Search to that level of nuance is not mission critical, I don't care what anyone says, you'll live if you wait another 15 minutes or even an hour. Their search has been terrible since their last major set of search changes where they overhauled it completely either way.