5 ms·
GitHub Actions is a seriously flawed CI system
This is a bit of a rant, but also heads up for anyone who is considering adopting it, and aimed at someone in Microsoft who can influence this project.
I am just going to bullet point some of the shortcomings that make it practically unusable for any serious deployment:
* You cannot view logs of ongoing jobs. If you navigate to a job view after a task has already started, all you are going to see is a spinner. The job might be stuck in a loop, there might be an error, etc. You won't know until the job ends.
* You cannot define which jobs have to _always_ run, i.e. setup and teardown. Suppose you have a matrix of test runners. You need to setup the test environment before jobs start and teardown after they finish. You have the ability to do that on the happy path, but if one of the shards fails and it needs to be retried, there is no way to tell CI to "you must re-run the setup step before retrying this job"
* You cannot restrict when CI jobs should run, e.g. Let's say you want to NOT run jobs for Draft PRs. You may try to add `if: github.event.pull_request.draft == false` but that has two serious drawbacks: 1) check will be reported as passed even though it didn't run 2) updating PR from Draft -> Ready does not trigger
* You cannot reliably cancel jobs. Sometimes jobs just get stuck and no amount of "Cancel job" requests do anything. The only thing that eventually stops them is the timeout (tip: set timeout for every step exactly for this reason)
* There is no built-in mechanism for retries.
* You cannot manually retry a particular job until every job in the workflow has finished.
This is a non-exhaustive list.
- jjgreen 3y agoYou could give GitLab CI a try ...
- gajus 3y agoI love GitLab CI. We are too deep into GitHub Actions to migrate out of it. For every bullet-point above we've built something in house to workaround it. Things work, but it took many many months to get to the point where it is usable.
- duped 3y agoEveryone I know who has spent serious time digging into GH actions has eventually spent an equal amount of time moving off it, because it sucks and is crazy expensive.
- zero_shift 3y agoMy big problem with it is lack of insights. When you scale dev you really need detailed stats on CI performance and bottlenecks
- xwalk 3y agoWe are working on that, starting with the workflow level (which workflows in an org are using the most time) and then intend to move to the next level down (which jobs or steps within workflows are consuming the most time and what are the trends over time).
- kypro 3y agoI worked at a place that used GitHub Actions quite recently. I thought it was quite neat and we never had a problem with it, although our CI was fairly simple. I always thought GitHub Actions was intended as simple CI solution for small Github projects. If you need more control and functionality, wouldn't you be better off using dedicated CI system?
- chatmasta 3y agoI prefer GitLab CI for raw, get-things-done style configuration. GitHub Actions invents its own world of abstractions that can be fun to play with but that ultimately reduce productivity.
- SkyPuncher 3y agoWe must not be using the same GitHub Actions because most of the list is very much things you can do. I do them almost daily. Further, it sounds like some of your problems are fundamentally incorrect designs in your CI/CD pipe. Not just specific to GitHub Actions.
- gajus 3y agoFound the MS PM
- pawelduda 3y ago> * You cannot view logs of ongoing jobs. If you navigate to a job view after a task has already started, all you are going to see is a spinner. The job might be stuck in a loop, there might be an error, etc. You won't know until the job ends. Don't understand this one. I'm checking this all the time, for instance test runners. Unless you meant older logs from before you opened the page - that one is indeed a bit annoying in some cases I also find it annoying that yaml config is docker-compose like but it's easy to run into random limitations or incompatibilities...
- gajus 3y ago> Don't understand this one. I'm checking this all the time, for instance test runners. Unless you meant older logs from before you opened the page - that one is indeed a bit annoying in some cases That
- roland35 3y agoAre there any good articles or overviews about GitHub actions versus other CI tools? It's surprisingly hard to Google for...
- xwalk 3y ago> aimed at someone in Microsoft who can influence this project Hello. I'm the head of product for Actions. Thank you for the comments and we take all customer feedback seriously. I wanted to let you know that we are working on the "you cannot view logs of ongoing jobs" issue. We have also recently shipped a "force cancel" feature (available via API right now until we can build a UI -- https://github.blog/changelog/2023-09-21-github-actions-force-cancel-workflows/ https://github.blog/changelog/2023-09-21-github-actions-forc...)
- gajus 3y agoLove it. Email gajus@gajus.com if you'd like to chat about our experience. We are running something north of 1,000 CPUs on GitHub Actions.