4 ms·

I've used GitHub Actions quite extensively now, across infrastructure automation, Python CI/CD, and iOS CI/CD, and while not perfect, it's the best platform I'v

I've used GitHub Actions quite extensively now, across infrastructure automation, Python CI/CD, and iOS CI/CD, and while not perfect, it's the best platform I've used for this stuff so far. Compared to Jenkins it needed far less maintenance. Compared to CircleCI it felt much easier to work with and to build reliable pipelines due to the locking primitives it provides, and compared to Semaphore I found it easier to understand how the pieces fit together. My criticism would mostly be about missing features, but the pace of development has been great and the only one I have left on my list is SSH debugging, which didn't end up being much of a blocker to our adoption anyway. As for reliability, nothing is perfect, but in my experience it's at-or-above the level that CircleCI provided, and far surpassed our in-house Jenkins server.

judge2020 5y ago
> My criticism would mostly be about missing features, My only real requested feature is one that's been open since early 2020 (this issue replaced the old 2020 issue) and has the status "Status: Q1 2022 – Jan-Mar" https://github.com/github/roadmap/issues/161 https://github.com/github/roadmap/issues/161 Without this, i'm relying on Azure actions to start/stop+deallocate a self-hosted runner, which adds ~1 minute to the job startup time.
ethomson 5y ago
Thanks for the feedback! I'm one of the PMs for GitHub Actions, and I appreciate this. Thinking about Actions as a set of primitives that you can compose is very much how I think about the product (and I think the other PMs as well) so I'm glad that resonates. We're always welcome to feedback, and we're continuing to invest and improve on the product, so I'm hopeful that we can address the features that you're missing.
- newman314 5y ago
  Here's my ask: * Setting up GHA is still a lot of "commit and hope for the best". I've resorted to having a sandbox repo just for experimentation/testing so that I don't overly pollute repos that I actually care about. It would be great to get more instrumentation to see what is going on. * I have a monorepo for Dockerfiles. It's quite annoying that I have to have separate invocations for different Dockerfiles in dependabot.yml. I should be able to specify /Dockerfile or /Dockerfile* as patterns for detection. The Dependabot invocation for GitHub Actions is a single entry and it would be great to have that. * I quite like Step Security's Harden Runner but it does require more work/invocations to get this set up. Maybe GH can work with them to more closely incorporate said functionality? * Make the cache bigger? I build a fair number of multi-arch containers and starting all of them at once tends to blow out the cache. * Given the interest around sigstore and SBOMs, maybe incorporate native capabilities to sign artifacts and generate SBOMS?
  - lapser 5y ago
    > Setting up GHA is still a lot of "commit and hope for the best". I've resorted to having a sandbox repo just for experimentation/testing so that I don't overly pollute repos that I actually care about. It would be great to get more instrumentation to see what is going on. There is act[0] which aims to let you run github actions locally via Docker. It isn't perfect but it does a decent job at it, and for the most part your pipeline can be run locally. After MS bought GH, I had hopes that they would build a tool to run action locally, but nothing yet. [0] https://github.com/nektos/act https://github.com/nektos/act
    - OJFord 5y ago
      I've had no luck reproducing problems in Actions with act, and the rest of the time have problems in act that I don't in Actions it seems. I like the idea and also would like something first-party, but I imagine it's hard and GitHub would want it to be less buggy than act is, and maybe they're trying but it's not there. Tbh even if it ran remotely in actual Actions, but just didn't show up in the repo UI, logged locally, that would be fine?
  - ethomson 5y ago
    Thanks. The "commit and hope for the best" problem really resonates with me. There are two great projects that might provide some pain relief - nektos/act or rhysd/actionlint. But I agree that commit-to-validate is probably the best strategy at the moment, which is deeply unfortunate. This is an area that I intend to improve in the future. As for the cache, we doubled it at the end of last year to 10GB. https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/ https://github.blog/changelog/2021-11-23-github-actions-cach..., but I can see how multi-arch images would be very large. Have you considered putting images into GitHub Container Registry instead of putting the layers into the cache? I'd love to understand if that is appropriate for your workflow, and if not, what the limitation there is. Appreciate the rest of the feedback, I'll pass it along to the appropriate teams.
- explorigin 5y ago
  From my perspectives, GHA is missing 2 things over CircleCI. A way to pause an action for approval, or a way to pull artifacts from other workflows. Both of these actions are _possible_ with an external service but painful to setup. I want to: create a terraform plan, approve it, and then deploy the specifically approved plan. That's not so difficult in CircleCI but is _painful_ in GHA.
  - zhfliz 5y ago
    fwiw, you can use approvals using environments: https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#required-reviewers https://docs.github.com/en/actions/deployment/targeting-diff...
    - explorigin 5y ago
      Sure, but that's actually worse than useless for my use-case. Image this, you have an action that publishes your plan to your PR (#1 - it's a biggish feature). It gets merged and goes to approval. Then people happen. PR #2 is addressing a customer-facing bug so it gets fast-tracked and rammed through before PR #1. Suddenly PR #1 is silently invalid. It _should_ be rejected at this point but the whole point of CI/CD is to save time and reduce the surface area for human mistakes.
      - zhfliz 5y ago
        specifically for your terraform example, wouldn't it make more sense to have the PR merged only when apply was successful? i'm not sure how well that can be represented in GH actions, but that would surely be the better option? you'll always risk some kind of race condition there, e.g. atlantis locks the project while something is planned but not applied to avoid such things from happening. this of course prevents having multiple PRs "ready" at the same time, you'd have to unlock the active PR lock to be able to implement another one.
        explorigin 5y ago
- OJFord 5y ago
  That's very much how I think about it too, which is why it frustrates me that I can't create canned workflows that apply to all my repos of a certain type (language specific linting and releasing say). I know I can create user/organisation templates, but all that does is put it in the UI chooser to create a commit to put it in the repo from the web. I want to do something like `include: OJFord/workflows/terraform-provider.yml` or `include: OJFord/workflows/rust.yml` Perhaps even better would be I don't even have to specify that in the repo, they just apply automatically to any which match a given pattern - named `terraform-provider-*` or having a file `Cargo.toml` say - but I realise that's probably too big a deviation from the way Actions works at this point.
  - a_t48 5y ago
    Could you stuff your actions definition into a sub module pointed at trunk?
    - OJFord 5y ago
      Interesting idea! Even if Actions will follow a submodule though (I doubt it tbh, it happens before any actions run of course so we have no control over that) you can't point it 'at trunk' as far as I know, they're always at a specific commit. (E.g. if you git submodule foreach git checkout master, your diff if you have one will be updating commit hashes, not -whatever +master. This is good for a lot of other reasons but doesn't help here.)
  - judge2020 5y ago
    This might be what you're looking for: https://github.com/github/roadmap/issues/74 https://github.com/github/roadmap/issues/74
- anecd0te 5y ago
chatmasta 5y ago
We use GitLab CI for internal code and GitHub Actions for public code, and are generally happy with both. GitHub Actions emphasizes reusability and composability around “actions” as installable packages suitable for a “marketplace” model. This works well, but once you deviate from a standard action, changing its behavior requires learning (or re-learning every six months) the GitHub Actions DSL to make it do what you want. This can be frustrating when it feels like you need to model your task to fit within an abstraction that didn’t help you in the first place. GitLab CI feels much more “raw” and closer to the metal – there is less emphasis on packaging code into reusable workflows, and more emphasis on the practicalities of operating robust CI pipelines. Yes, you could have a different runner for every task, and “package” CI code into Docker images similarly to how GitHub Actions packages it into workflows. But the service isn’t designed around that idea. And for internal pipelines, that feels like a needless abstraction. It’s nice to keep pipelines simple and robust. Put another way: there is much less “magic” in GitLab CI than in GitHub Actions. In many ways GitHub Actions feels more like a toy than a tool. This is generally a good thing, IMO, and overall I’d say the tradeoffs are worth it for both services, especially given the different contexts (internal vs. public code) in which we use them.
- danpalmer 5y ago
  I forgot to mention GitLab CI, but in my experience, much like with Travis CI, it is as you say lower level. I do not find that to be a good thing. Rather than lower level being faster as it is with code, generally with CI we want to raise the abstraction level so that it's easier to make use of things that make CI run faster – parallelism in workflows, caching etc. As for reliability, I also found GitLab CI lacked the concurrency control caching primitives necessary to trust continuous delivery pipelines. I found you could either write a slow correct pipeline, or a faster but fundamentally unsafe one.
telotortium 5y ago
> SSH debugging Have you seen https://github.com/marketplace/actions/debugging-with-ssh https://github.com/marketplace/actions/debugging-with-ssh? (Maybe this only works with public Github - that's all I use). I add an `if: ${{ failure() }}` to only spawn the SSH when my job fails and use that to debug CI issues.
WhyNotHugo 5y ago
It's definitely more pleasant to use than Jenkins, but Jenkins is as painful as it gets. 90% of the times it's some ancient setup that nobody knows how it works. And the amount of hands-on required to keep in working is waaaay higher than other CI pipelines. Circle CI is also not very nice at all. Things like GitLab CI or Sourcehut Builds are sooo much simpler, lighter and faster in comparison.
- danpalmer 5y ago
  Jenkins sucks from a UI perspective, and from a maintenance perspective, but it's very powerful in ways that most other CIs are not. The classic example for us was concurrency control. We did continuous delivery, and so it's critical that you're not trying to release the `master` branch multiple times concurrently, especially when database migrations might happen, etc. With Jenkins you could quite easily say "only 1 of this job at a time", but this wasn't (at the time) a feature of CircleCI, GitLab, or Travis. GitHub Actions added it about 2 years ago I think, and Semaphore has had something similar for a while. While I agree Jenkins is painful, having this feature and the reliability it brought reduced pain significantly. CircleCI not having this feature, for us, caused a lot of pain.

judge2020 5y ago

> My criticism would mostly be about missing features, My only real requested feature is one that's been open since early 2020 (this issue replaced the old 2020 issue) and has the status "Status: Q1 2022 – Jan-Mar" https://github.com/github/roadmap/issues/161 https://github.com/github/roadmap/issues/161 Without this, i'm relying on Azure actions to start/stop+deallocate a self-hosted runner, which adds ~1 minute to the job startup time.

ethomson 5y ago

Thanks for the feedback! I'm one of the PMs for GitHub Actions, and I appreciate this. Thinking about Actions as a set of primitives that you can compose is very much how I think about the product (and I think the other PMs as well) so I'm glad that resonates. We're always welcome to feedback, and we're continuing to invest and improve on the product, so I'm hopeful that we can address the features that you're missing.

newman314 5y ago
Here's my ask: * Setting up GHA is still a lot of "commit and hope for the best". I've resorted to having a sandbox repo just for experimentation/testing so that I don't overly pollute repos that I actually care about. It would be great to get more instrumentation to see what is going on. * I have a monorepo for Dockerfiles. It's quite annoying that I have to have separate invocations for different Dockerfiles in dependabot.yml. I should be able to specify /Dockerfile or /Dockerfile* as patterns for detection. The Dependabot invocation for GitHub Actions is a single entry and it would be great to have that. * I quite like Step Security's Harden Runner but it does require more work/invocations to get this set up. Maybe GH can work with them to more closely incorporate said functionality? * Make the cache bigger? I build a fair number of multi-arch containers and starting all of them at once tends to blow out the cache. * Given the interest around sigstore and SBOMs, maybe incorporate native capabilities to sign artifacts and generate SBOMS?
- lapser 5y ago
  > Setting up GHA is still a lot of "commit and hope for the best". I've resorted to having a sandbox repo just for experimentation/testing so that I don't overly pollute repos that I actually care about. It would be great to get more instrumentation to see what is going on. There is act[0] which aims to let you run github actions locally via Docker. It isn't perfect but it does a decent job at it, and for the most part your pipeline can be run locally. After MS bought GH, I had hopes that they would build a tool to run action locally, but nothing yet. [0] https://github.com/nektos/act https://github.com/nektos/act
  - OJFord 5y ago
    I've had no luck reproducing problems in Actions with act, and the rest of the time have problems in act that I don't in Actions it seems. I like the idea and also would like something first-party, but I imagine it's hard and GitHub would want it to be less buggy than act is, and maybe they're trying but it's not there. Tbh even if it ran remotely in actual Actions, but just didn't show up in the repo UI, logged locally, that would be fine?
- ethomson 5y ago
  Thanks. The "commit and hope for the best" problem really resonates with me. There are two great projects that might provide some pain relief - nektos/act or rhysd/actionlint. But I agree that commit-to-validate is probably the best strategy at the moment, which is deeply unfortunate. This is an area that I intend to improve in the future. As for the cache, we doubled it at the end of last year to 10GB. https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/ https://github.blog/changelog/2021-11-23-github-actions-cach..., but I can see how multi-arch images would be very large. Have you considered putting images into GitHub Container Registry instead of putting the layers into the cache? I'd love to understand if that is appropriate for your workflow, and if not, what the limitation there is. Appreciate the rest of the feedback, I'll pass it along to the appropriate teams.
explorigin 5y ago
From my perspectives, GHA is missing 2 things over CircleCI. A way to pause an action for approval, or a way to pull artifacts from other workflows. Both of these actions are _possible_ with an external service but painful to setup. I want to: create a terraform plan, approve it, and then deploy the specifically approved plan. That's not so difficult in CircleCI but is _painful_ in GHA.
- zhfliz 5y ago
  fwiw, you can use approvals using environments: https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#required-reviewers https://docs.github.com/en/actions/deployment/targeting-diff...
  - explorigin 5y ago
    Sure, but that's actually worse than useless for my use-case. Image this, you have an action that publishes your plan to your PR (#1 - it's a biggish feature). It gets merged and goes to approval. Then people happen. PR #2 is addressing a customer-facing bug so it gets fast-tracked and rammed through before PR #1. Suddenly PR #1 is silently invalid. It _should_ be rejected at this point but the whole point of CI/CD is to save time and reduce the surface area for human mistakes.
    - zhfliz 5y ago
      specifically for your terraform example, wouldn't it make more sense to have the PR merged only when apply was successful? i'm not sure how well that can be represented in GH actions, but that would surely be the better option? you'll always risk some kind of race condition there, e.g. atlantis locks the project while something is planned but not applied to avoid such things from happening. this of course prevents having multiple PRs "ready" at the same time, you'd have to unlock the active PR lock to be able to implement another one.
      - explorigin 5y ago
OJFord 5y ago
That's very much how I think about it too, which is why it frustrates me that I can't create canned workflows that apply to all my repos of a certain type (language specific linting and releasing say). I know I can create user/organisation templates, but all that does is put it in the UI chooser to create a commit to put it in the repo from the web. I want to do something like `include: OJFord/workflows/terraform-provider.yml` or `include: OJFord/workflows/rust.yml` Perhaps even better would be I don't even have to specify that in the repo, they just apply automatically to any which match a given pattern - named `terraform-provider-*` or having a file `Cargo.toml` say - but I realise that's probably too big a deviation from the way Actions works at this point.
- a_t48 5y ago
  Could you stuff your actions definition into a sub module pointed at trunk?
  - OJFord 5y ago
    Interesting idea! Even if Actions will follow a submodule though (I doubt it tbh, it happens before any actions run of course so we have no control over that) you can't point it 'at trunk' as far as I know, they're always at a specific commit. (E.g. if you git submodule foreach git checkout master, your diff if you have one will be updating commit hashes, not -whatever +master. This is good for a lot of other reasons but doesn't help here.)
- judge2020 5y ago
  This might be what you're looking for: https://github.com/github/roadmap/issues/74 https://github.com/github/roadmap/issues/74
anecd0te 5y ago

chatmasta 5y ago

We use GitLab CI for internal code and GitHub Actions for public code, and are generally happy with both. GitHub Actions emphasizes reusability and composability around “actions” as installable packages suitable for a “marketplace” model. This works well, but once you deviate from a standard action, changing its behavior requires learning (or re-learning every six months) the GitHub Actions DSL to make it do what you want. This can be frustrating when it feels like you need to model your task to fit within an abstraction that didn’t help you in the first place. GitLab CI feels much more “raw” and closer to the metal – there is less emphasis on packaging code into reusable workflows, and more emphasis on the practicalities of operating robust CI pipelines. Yes, you could have a different runner for every task, and “package” CI code into Docker images similarly to how GitHub Actions packages it into workflows. But the service isn’t designed around that idea. And for internal pipelines, that feels like a needless abstraction. It’s nice to keep pipelines simple and robust. Put another way: there is much less “magic” in GitLab CI than in GitHub Actions. In many ways GitHub Actions feels more like a toy than a tool. This is generally a good thing, IMO, and overall I’d say the tradeoffs are worth it for both services, especially given the different contexts (internal vs. public code) in which we use them.

danpalmer 5y ago
I forgot to mention GitLab CI, but in my experience, much like with Travis CI, it is as you say lower level. I do not find that to be a good thing. Rather than lower level being faster as it is with code, generally with CI we want to raise the abstraction level so that it's easier to make use of things that make CI run faster – parallelism in workflows, caching etc. As for reliability, I also found GitLab CI lacked the concurrency control caching primitives necessary to trust continuous delivery pipelines. I found you could either write a slow correct pipeline, or a faster but fundamentally unsafe one.

telotortium 5y ago

> SSH debugging Have you seen https://github.com/marketplace/actions/debugging-with-ssh https://github.com/marketplace/actions/debugging-with-ssh? (Maybe this only works with public Github - that's all I use). I add an `if: ${{ failure() }}` to only spawn the SSH when my job fails and use that to debug CI issues.

WhyNotHugo 5y ago

It's definitely more pleasant to use than Jenkins, but Jenkins is as painful as it gets. 90% of the times it's some ancient setup that nobody knows how it works. And the amount of hands-on required to keep in working is waaaay higher than other CI pipelines. Circle CI is also not very nice at all. Things like GitLab CI or Sourcehut Builds are sooo much simpler, lighter and faster in comparison.

danpalmer 5y ago
Jenkins sucks from a UI perspective, and from a maintenance perspective, but it's very powerful in ways that most other CIs are not. The classic example for us was concurrency control. We did continuous delivery, and so it's critical that you're not trying to release the `master` branch multiple times concurrently, especially when database migrations might happen, etc. With Jenkins you could quite easily say "only 1 of this job at a time", but this wasn't (at the time) a feature of CircleCI, GitLab, or Travis. GitHub Actions added it about 2 years ago I think, and Semaphore has had something similar for a while. While I agree Jenkins is painful, having this feature and the reliability it brought reduced pain significantly. CircleCI not having this feature, for us, caused a lot of pain.