5 ms·
GitHub Actions Outage
- jzebedee 2y agoAll of Azure Central US is having issues. [1] [1] https://azure.status.microsoft/en-us/status https://azure.status.microsoft/en-us/status
- orliesaurus 2y agoOuch, there goes our build pipeline
- p1necone 2y agoAzure DevOps is totally down too, to the extent that I can't even git push to a remote hosted there (get a 503 response). Something very critical has fallen over. Edit: I don't think this is for everyone though - maybe only if your ADO instance happens to be hosted in US Central?
- xyzzy4747 2y ago[flagged]
- Trasmatta 2y agoSSH and shell scripts that also constantly break and require maintenance? I don't have any nostalgia for those days either.
- hu3 2y agoI agree with the sentiment that we should not rely on clouds to ship software. However, the server we would SSH to to probably has less uptime than Azure.
- deleted 2y ago[deleted]
- willsr 2y agoIssues with VSCode Marketplace as well.
- aabhay 2y agoAre self hosted runners down as well?
- suryao 2y agoCan confirm that self hosted runners are seeing issues at WarpBuild. Just another Thursday :shrug: https://warpbuild.instatus.com/ https://warpbuild.instatus.com/
- jacobwg 2y agoKinda - what we're seeing at Depot (https://status.depot.dev/clyrvud6i57402igofm6jtb7id https://status.depot.dev/clyrvud6i57402igofm6jtb7id) is that the APIs for managing Actions runners or retrieving work are receiving rate limit errors, interestingly missing the regular rate limit status headers[1]. We do have some jobs processing, but overall the GitHub API and Actions control plane seem very overloaded by the outage. [1] - https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api#checking-the-status-of-your-rate-limit https://docs.github.com/en/rest/using-the-rest-api/rate-limi...
- behnamoh 2y agoInstead of saying means things about GA, can anyone chip in about ways they've made their work more resilient and fault tolerant in case such incidents happen?
- haykuro 2y ago[flagged]
- cr125rider 2y agoWe run Bitbucket pipelines, but a similar notion applies. Everything we do is in a container. If we wanted to run our CI manually, we can just run our steps as a script on anyone’s workstation (given keys/permission). Staying away from the proprietary run environments and being easily to run them ourselves has saved us a number of times. Especially when running on Atlassian where things go down often.
- TillE 2y agoOf course there are plenty of self-hosting options if you want that responsibility, but a very rare outage of CI/CD doesn't seem like something that's worth spending resources planning around. On the scale of possible disruptions, it's pretty minor.
- synicalx 2y agoWe looked into it, very briefly, and decided that a few hours or even a whole day of downtime per year wasn't really worth the time and cost it would take to mitigate this in our case. If our pipelines happen to fall over, and it also happens to occur during our working hours then the solution for us is to either work on something else or head down to the pub.
- jesseduffield 2y agoStill down for me
- dfgdgf 2y ago[flagged]