3 ms·

Fun read about Azure and having 173 agents running a node: https://isolveproblems.substack.com/p/how-microsoft-vaporized-a-trillion https://isolveproblems.subst

Fun read about Azure and having 173 agents running a node: https://isolveproblems.substack.com/p/how-microsoft-vaporized-a-trillion https://isolveproblems.substack.com/p/how-microsoft-vaporize... Probably just a coincidence that Github started to have issues after beginning their move to Azure at the end of last year.

Syntaf 1mo ago
One of my favorite articles posted here for this year, it's a great read and really gives you an idea of just how dysfunctional azure is.
caust1c 1mo ago
Azure's going to suffocate github. I'm curious to see what's next. Will self-hosting the code repository come back in vogue or will another social-coding platform take off?
- jscott817 1mo ago
  My small org has definitely had internal discussions around self-hosting gitlab. We'll see what happens.
  - vinnymac 1mo ago
    If you're actively considering self hosting, I recommend giving forgejo a try. The experience is much more similar to what GitHub offers, actions API is nearly identical to name one similarity of which there are many. Moving essentially becomes one prompt and a coffee later for a small team.
    - hosh 1mo ago
      If people are happy with Actions API, that sounds great. I’m not happy with the Actions API. I think Gitlab’s cicd design is much better, and I’m not fighting it all the time when I use it.
      - dijksterhuis 1mo ago
        i'll echo this sentiment. historically there's been a lot more flexibility with GL CI yaml syntax, including real yaml anchors https://frenck.dev/github-actions-yaml-anchors-aliases-merge-keys/ https://frenck.dev/github-actions-yaml-anchors-aliases-merge... sys-admin-ing gitlab can be a bit of a maze of web menus tho.
        spockz 1mo ago
        What do you need real anchors for? Sharing of pipelines? Maybe I’ve been burned too much by pipeline maintenance (because we didn’t have yaml anchors?) but I rather have builds defined in make /bazel/etc than in yaml. So the only thing the pipeline does is optionally restoring caches, kicking off the build system, uploading PR validation results, and saving cache. Pushing artifact etc is all done from inside the build system. There is no “setup” like installing packages because we make the build image seperately.
        deleted 1mo ago
        [deleted]
        vinnymac 1mo ago
        I find this recommendation funny, because I completely agree from a sysadmin perspective. I used to use Gitlab at work, and small teams would run into so many footguns with CI that we had to throw up guardrails to prevent mistakes. Far too many links to gitlab issues that were not fixed even after >8+ years of being open ended up biting us. With GHA I haven't had that experience, and same for all of my self hosted Forgejo instances. I used to hate using GHA from about ~2018 to 2021, but they've fixed a lot of things I disliked since then.
  - stephenway 1mo ago
    Curious what's driving the self-hosting discussion most, reliability, control, or compliance? Disclosure, I run a managed forgejo service
- brazukadev 1mo ago
  probably not. The people that have done it before or willing to do it now is probably a very % of the commit volume, they leaving wouldn't change much, probably not gonna even move the exponential growth needle.
- pocksuppet 1mo ago
  We should worry less about what everyone else uses, and more about what we use. I'm self-hosting Gitea and thinking of upgrading to Forgejo. What are you using?
  - caust1c 1mo ago
    I actually just installed forgejo in My Home lab and loving it. Thinking about moving the companys code hosting to it next.
    - stephenway 1mo ago
      If you move the company too, what would make you pay someone to run forgejo instead of self-hosting it? Disclosure, I run Fjord, a managed dedicated forgejo provider.
  - worik 1mo ago
    Yes But GitHub, in its heyday, was a centre. Created a sense of "community" It has taken a while, but since MS bought it it has been shirking that. I expected honest enshittification, but instead it has been technical collapse What ever. IMO we need a federation protocol for Git that can rebuild some sort of "community", but on solid foundations. So we can find one another on our self hosted instances
    - pocksuppet 1mo ago
      There was never a "sense of community", there was only low friction on issue reporting and especially, commenting on an issue you had nothing else to do with. Every time an issue hit HN it would be flooded with bystander comments.
  - waz0wski 1mo ago
    I've moved from gitlab -> gitea+drone -> forgejo+woodpecker Works great for small-medium scale
- anvuong 1mo ago
  I used to work in a small independent team of 30 people within a large corp, half of which was dev. We used to run our own gitlab on-prem, our CI/CD was also on-prem. It worked perfectly, never had down time, devops guy could configure them on-demand to our needs. Me (and some other guys) also jumped in times to times to help (mostly just ssh into the servers for health check, disk partition, etc.). Then we grew (the biz team, dev was the same) and some new PMs with fancy Ivy League degrees came in and pushed for on-cloud Bitbucket. Things went to shit pretty fast after that ... Our codebase was only a few hundred thousands lines, there was only like hundreds of commits per day, the servers our git + CI/CD lived on never saturated ...
  - swozey 1mo ago
    PMs deciding software infrastructure over the dev teams that the dev teams use, is wild.
    - mobelkh 1mo ago
      they want that jira <-> bitbucket integration
    - anvuong 1mo ago
      From my experiences devs in large corp usually don't have a say in what kind of software infrastructure they can use ...
      - swozey 1mo ago
        I've mostly worked for "engineering lead" corps and a PM has only ever affected outside software sold to users except for JIRA. There's no chance a PM would be able to affect the internal code storage/ci-cd systems at anywhere I've worked in the past unless they went on some proselytizing war path and convinced some senior lead engs to convince the rest of the org to accept it. In the example, in my 20 year career, I've seen bitbucket in use once. I'm the one who usually manages that stuff since I'm an infra eng.
- Etheryte 1mo ago
  To be honest, I'm very much surprised Amazon hasn't eaten Microsoft's lunch here. Offer out-of-the-box AWS instances with Gitlab, SLAs, backups and the works, should be straightforward.
outworlder 1mo ago
> Probably just a coincidence that Github started to have issues after beginning their move to Azure at the end of last year. Right. I'd like to think that Azure has improved in a meaningful way in the past half a decade or so... but it has not. Maybe less inexplicable 400 errors in random API calls, I guess. And yet Microsoft keeps posting record growth. Unbelievable.
whoamii 1mo ago
Keep in mind the seniority of the author. This was not written by a staff+ engineer.
slices 1mo ago
"fun" I thought I was cynical enough about goings on at Microsoft / Azure, but apparently not.
timcambrant 1mo ago
I've heard that companies that run enterprise Github instances on-premises have almost equally poor availability compared to github.com. There are probably plenty of reasons why they have so many outages, though I agree one of them is probably Azure. The outages started around the same time as the number of pushed commits skyrocketed, and the number of Dependabot vulnerabilities started growing as well, which in its turn increases the amount of PRs and Actions activity. Also, LLMs regularly import dependencies that are already out of date, meaning that when doing agentic development, one commit often results in several Dependabot PRs being created instantly.

Syntaf 1mo ago

One of my favorite articles posted here for this year, it's a great read and really gives you an idea of just how dysfunctional azure is.

caust1c 1mo ago

Azure's going to suffocate github. I'm curious to see what's next. Will self-hosting the code repository come back in vogue or will another social-coding platform take off?

jscott817 1mo ago
My small org has definitely had internal discussions around self-hosting gitlab. We'll see what happens.
- vinnymac 1mo ago
  If you're actively considering self hosting, I recommend giving forgejo a try. The experience is much more similar to what GitHub offers, actions API is nearly identical to name one similarity of which there are many. Moving essentially becomes one prompt and a coffee later for a small team.
  - hosh 1mo ago
    If people are happy with Actions API, that sounds great. I’m not happy with the Actions API. I think Gitlab’s cicd design is much better, and I’m not fighting it all the time when I use it.
    - dijksterhuis 1mo ago
      i'll echo this sentiment. historically there's been a lot more flexibility with GL CI yaml syntax, including real yaml anchors https://frenck.dev/github-actions-yaml-anchors-aliases-merge-keys/ https://frenck.dev/github-actions-yaml-anchors-aliases-merge... sys-admin-ing gitlab can be a bit of a maze of web menus tho.
      - spockz 1mo ago
        What do you need real anchors for? Sharing of pipelines? Maybe I’ve been burned too much by pipeline maintenance (because we didn’t have yaml anchors?) but I rather have builds defined in make /bazel/etc than in yaml. So the only thing the pipeline does is optionally restoring caches, kicking off the build system, uploading PR validation results, and saving cache. Pushing artifact etc is all done from inside the build system. There is no “setup” like installing packages because we make the build image seperately.
        deleted 1mo ago
        [deleted]
      - vinnymac 1mo ago
        I find this recommendation funny, because I completely agree from a sysadmin perspective. I used to use Gitlab at work, and small teams would run into so many footguns with CI that we had to throw up guardrails to prevent mistakes. Far too many links to gitlab issues that were not fixed even after >8+ years of being open ended up biting us. With GHA I haven't had that experience, and same for all of my self hosted Forgejo instances. I used to hate using GHA from about ~2018 to 2021, but they've fixed a lot of things I disliked since then.
- stephenway 1mo ago
  Curious what's driving the self-hosting discussion most, reliability, control, or compliance? Disclosure, I run a managed forgejo service
brazukadev 1mo ago
probably not. The people that have done it before or willing to do it now is probably a very % of the commit volume, they leaving wouldn't change much, probably not gonna even move the exponential growth needle.
pocksuppet 1mo ago
We should worry less about what everyone else uses, and more about what we use. I'm self-hosting Gitea and thinking of upgrading to Forgejo. What are you using?
- caust1c 1mo ago
  I actually just installed forgejo in My Home lab and loving it. Thinking about moving the companys code hosting to it next.
  - stephenway 1mo ago
    If you move the company too, what would make you pay someone to run forgejo instead of self-hosting it? Disclosure, I run Fjord, a managed dedicated forgejo provider.
- worik 1mo ago
  Yes But GitHub, in its heyday, was a centre. Created a sense of "community" It has taken a while, but since MS bought it it has been shirking that. I expected honest enshittification, but instead it has been technical collapse What ever. IMO we need a federation protocol for Git that can rebuild some sort of "community", but on solid foundations. So we can find one another on our self hosted instances
  - pocksuppet 1mo ago
    There was never a "sense of community", there was only low friction on issue reporting and especially, commenting on an issue you had nothing else to do with. Every time an issue hit HN it would be flooded with bystander comments.
- waz0wski 1mo ago
  I've moved from gitlab -> gitea+drone -> forgejo+woodpecker Works great for small-medium scale
anvuong 1mo ago
I used to work in a small independent team of 30 people within a large corp, half of which was dev. We used to run our own gitlab on-prem, our CI/CD was also on-prem. It worked perfectly, never had down time, devops guy could configure them on-demand to our needs. Me (and some other guys) also jumped in times to times to help (mostly just ssh into the servers for health check, disk partition, etc.). Then we grew (the biz team, dev was the same) and some new PMs with fancy Ivy League degrees came in and pushed for on-cloud Bitbucket. Things went to shit pretty fast after that ... Our codebase was only a few hundred thousands lines, there was only like hundreds of commits per day, the servers our git + CI/CD lived on never saturated ...
- swozey 1mo ago
  PMs deciding software infrastructure over the dev teams that the dev teams use, is wild.
  - mobelkh 1mo ago
    they want that jira <-> bitbucket integration
  - anvuong 1mo ago
    From my experiences devs in large corp usually don't have a say in what kind of software infrastructure they can use ...
    - swozey 1mo ago
      I've mostly worked for "engineering lead" corps and a PM has only ever affected outside software sold to users except for JIRA. There's no chance a PM would be able to affect the internal code storage/ci-cd systems at anywhere I've worked in the past unless they went on some proselytizing war path and convinced some senior lead engs to convince the rest of the org to accept it. In the example, in my 20 year career, I've seen bitbucket in use once. I'm the one who usually manages that stuff since I'm an infra eng.
Etheryte 1mo ago
To be honest, I'm very much surprised Amazon hasn't eaten Microsoft's lunch here. Offer out-of-the-box AWS instances with Gitlab, SLAs, backups and the works, should be straightforward.

outworlder 1mo ago

> Probably just a coincidence that Github started to have issues after beginning their move to Azure at the end of last year. Right. I'd like to think that Azure has improved in a meaningful way in the past half a decade or so... but it has not. Maybe less inexplicable 400 errors in random API calls, I guess. And yet Microsoft keeps posting record growth. Unbelievable.

whoamii 1mo ago

Keep in mind the seniority of the author. This was not written by a staff+ engineer.

slices 1mo ago

"fun" I thought I was cynical enough about goings on at Microsoft / Azure, but apparently not.

timcambrant 1mo ago

I've heard that companies that run enterprise Github instances on-premises have almost equally poor availability compared to github.com. There are probably plenty of reasons why they have so many outages, though I agree one of them is probably Azure. The outages started around the same time as the number of pushed commits skyrocketed, and the number of Dependabot vulnerabilities started growing as well, which in its turn increases the amount of PRs and Actions activity. Also, LLMs regularly import dependencies that are already out of date, meaning that when doing agentic development, one commit often results in several Dependabot PRs being created instantly.