9 ms·

GitHub Actions Are a Problem

• acjohnson55 3y ago
  I love GitHub Actions for basic CI/CD use cases, but I agree with the criticisms that testing locally is a major pain point and coding as a YAML DSL is a minor pain point.
  • digitalsushi 3y ago
    I mean to me it's no different than a video game that's "online only". And I know how much I love that.
• polotics 3y ago
  What I always guessed: if you don't want to bite the bullet of trusting SRE scripters (with an s) to the role of building the CD that actually makes sense for your use case, why do you expect some random company will have more of your interest at heart? At least with Joe's python deployment scripts, Jane can do a code review. So much of cloud CI/CD lingo translates to "believe in magic, and also our bigness-based authority" when actually proper software deploys itself properly without making a fuss or glueing your fingers with sticky Yaml...
• CptKriechstrom 3y ago
  I've been migrating some Workflows to dagger.io. So far it's amazing.
  • redhale 3y ago
    Thanks for sharing, I was unaware of dagger.io! I'm curious now -- are there any other alternatives with similar philosophy/benefits?
    • maestrae 3y ago
      There's earthly.dev but like previous commenters, I went with dagger because i prefer its Apache 2.0 license over earthly's BSL.
      • adamgordonbell 3y ago
        Earthly is MPL 2.0 now. I work for earthly and people didn't like BSL so we listened and changed.
        • paradox460 3y ago
          Earthly is legitimately one of the coolest projects I wish I could use. But entrenched velocity in existing systems makes it a difficult pitch
          • adamgordonbell 3y ago
            That's great to here! Sometimes people find just starting with a messy part of the build process, or starting to replace dockerfiles here and there can add value, without having to change the world.
          • marcosnils 3y ago
            Hey, Dagger employee here. This exactly is one of the the main design decisions of Dagger. We're not expecting companies and/or projects to perform a full migration to it, that's why we generally recommend starting by wrapping different parts of your pipeline and move on from there. Unlike other solutions, Dagger allows you to keep using your existing tooling and reuse Dockerfiles (https://docs.dagger.io/quickstart/429462/build-dockerfile/#reuse-existing-dockerfiles https://docs.dagger.io/quickstart/429462/build-dockerfile/#r...) while slowly transition your pipelines to code.
      • redhale 3y ago
        Thanks for sharing, will definitely check this out!
    • tao_at_garden 3y ago
      Yes, there's us over at https://garden.io https://garden.io! We're big believers in pipelines that run anywhere. I even made a short little video that should give you the gist. [1] Some of the short-list of differences: we use YAML for our configuration language, Dagger can use full-fat languages to define its pipelines. Our feature scope is broader: you can use us to vend IDP-like stacks to your developers if you're a Platform Team; we make development with remote Kubernetes clusters very easy, including all the remote image builds; and we have a number of integrations so you can bring your IaC tool of choice (Pulumi, Terraform) into your pipeline and set up service -> infra dependencies. [1] https://www.youtube.com/watch?v=JFnan6s2cDg https://www.youtube.com/watch?v=JFnan6s2cDg
      • redhale 3y ago
        Thanks for sharing, will definitely check this out!
  • andthat 3y ago
    Same here, it's so liberating! I'm also using a Magefile to run the pipelines (written in Go) locally.
• ahofmann 3y ago
  This are the reasons I don't use GitHub actions. I don't want to add another hard dependency into my development environment and testing actions looks like a very slow feedback loop, very time consuming.
• simonw 3y ago
  If you want the ability to run as much of your GitHub Actions locally as possible (which you should! Testing via pushing commits is painful) you can arrange your actions so the majority of the logic lives in shell scripts (or Python scripts or whatever) that are called by the workflow. Then you can run those scripts locally and iterate in them before you try them on GitHub Actions. I agree: Actions would be more useful if I could run the entire stack locally (an official implementation, not a community-maintained clone). But it's not a big enough inconvenience for me to care very much.
  • mplewis 3y ago
    Incidentally, this is sort of how the Earthly runner serves to provide you local actions that also work in CI.
  • simcop2387 3y ago
    Isn't that exactly what the self hosted runners are? https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners https://docs.github.com/en/actions/hosting-your-own-runners/... Then you configure the repo settings to only use the self hosted ones.
    • jahsome 3y ago
      If you're suggesting the self-hosted runners are equivalent to running locally, then no, self-hosted runners still require a commit + push and runs remotely with the added overhead of the job being queued, reported, etc. The dream is to have something like a container _locally_ and be able to run something like: > gh actions run
  • dinkleberg 3y ago
    This is the way. It also lets you pretty much seamlessly switch CI tools at any time.
    • jahnu 3y ago
      I third this opinion: another reason it’s important is it allows you to do your build in an emergency when CI is down for whatever reason. CI automation is for me the thing that replaces me running scripts one by one (and reporting/deploying results) It is not the thing that does the building/testing/deploying. That’s the scripts that are hopefully written in a debuggable, portable, language.
• mb7733 3y ago
  My strategy is just to keep the yaml to a minimum and call python scripts within the action to do anything complicated. The GHA yaml is just used for defining when the action runs, and on what architectures, etc. It's worked well for me.
  • sametmax 3y ago
    Same. My local astk runner does the heavy lifting.
    • mst 3y ago
      Was that a typo for task or is 'astk' a punnily named task runner in its own right? (sorry for the dumbass question but it could go either way)
• hardwaresofton 3y ago
  GHA is "good enough" (GitLab CI was/is much better IMO), and it'll likely never go away because of how dominant GitHub is. Also, a simpler solution that takes a bit more discipline is writing all your CI tasks as scripts/makefile|justfile targets/etc and using containers to run them in GHA. IIRC containers are a little slower than running scripts in the VM directly (more than it must necessarily be), and also cross-platform builds might be slightly more complicated...
  • XorNot 3y ago
    Gitlab CI desperately needs backwards dependencies. A huge number of my remaining problems with it would be solved if I could subscribe to upstream outputs from my own pipelines.
    • mdaniel 3y ago
      I'm aware it says Premium but FWIW it does seem that exists since 12.8: https://docs.gitlab.com/ee/ci/pipelines/#trigger-a-pipeline-when-an-upstream-project-is-rebuilt https://docs.gitlab.com/ee/ci/pipelines/#trigger-a-pipeline-... I can also appreciate that the devil's in the details, so all its current constraints may not cover your use case
      • XorNot 3y ago
        So this seems to be enabled on the edition I'm using at work, but the limit is bizarrely "two pipelines". Which I already exceed since I have projects using git-dependencies in Python with 3+ repo dependencies. But it's also a weird project setting not a .gitlab-ci one.
• gavinhoward 3y ago
  It's scary how closed-source our development platforms are. That's why building a fully-self-contained VCS, one that includes CI, is on my roadmap, but I fear the open part of our industry will fail before I finish.
  • bogwog 3y ago
    Github is pretty much the only major one that's closed afaik. Gitlab, Gitea/Forgejo, and SourceHut are all great existing alternatives that are open source. It would probably be better to volunteer your time to one of those than to build another one.
    • gavinhoward 3y ago
      Isn't GitLab partially closed? Didn't Gitea suffer a hostile takeover recently? SourceHut seems to be alright, but the founder hates me.
      • bogwog 3y ago
        * Gitlab EE (enterprise edition) is closed, but Gitlab CE (community edition) is open source (https://gitlab.com/gitlab-org/gitlab-foss/ https://gitlab.com/gitlab-org/gitlab-foss/) * I didn't follow the Gitea drama too closely, but my understanding is that Forgejo was a fork born out of that situation * I've heard the SourceHut guy is a controversial figure, so avoiding it because of that isn't unreasonable. I will just say that "spite forks" tend not to last very long EDIT: actually, ignore my pointless negativity. I realize now that there really aren't that many open source code forges out there, so if you're willing to make one, then go for it!
        • gavinhoward 3y ago
          Thank you for your encouragement. :) You're very right about "spite forks"; I'm explicitly avoiding a fork of SourceHut because of that. I'm also going to be building the VCS from the ground up to solve some problems with Git; otherwise, why would anyone switch to my CI?
• dventimi 3y ago
  > Defining a language to run scripts in parallel with some constraints is actually quite easy if you just pass the script to an existing interpreter and let the language mainly deal with the constraints around order and parallelism (and code reuse) Isn't this Make?
  • rr808 3y ago
    Right I like doing everything in Bazel instead of make now but still the same idea to keep it independent of CICD.
    • javier2 3y ago
      Bazel? How many hours per year are you allowed to spend on CI? :D
      • rr808 3y ago
        Lol our company uses it a lot so I use it a bit. I never learned make properly so I figure I might as well ignore it for now.
• zzzeek 3y ago
  Github Actions gives me literally free server time across an extremely wide range of OS'es that I dont have to worry about at all, including Windows and OSX, which I therefore dont have to deal with in any way, buy any license keys, none of that. It's nothing short of miraculous for us as it's how Python projects can have binary wheel files for dozens of OSes and Python versions: https://pypi.org/project/SQLAlchemy/#files https://pypi.org/project/SQLAlchemy/#files . This task remained impossible for years (to be clear: because I don't have a server farm, or Windows licenses, or however you'd run OSX on a headless server, or any kind of resources to fund / manage dozens of images and keep them running, or any of that) until GH Actions made it possible. Now is this all part of Microsoft's evil plan ? It probably is! But unless someone else wants to give me a free server farm that includes Windows / OSX up and running without me paying anything / writing containers / storing images / etc. I dont see this aspect of Github actions losing any popularity.
  • Galanwe 3y ago
    > This task remained impossible for years until GH Actions made it possible. Every major distribution used build farms long before GitHub (and git) existed...
    • mplewis 3y ago
      Yes, but I don’t have a build farm.
      • throw555chip 3y ago
        Vendor lock-in hence the evil plan: https://dev.to/pwd9000/how-github-actions-can-improve-cicd-and-compares-with-azure-pipelines-3bh https://dev.to/pwd9000/how-github-actions-can-improve-cicd-a... "While GitHub Actions is an open platform, it is still tied to the GitHub ecosystem. This can be a concern for teams who want to maintain flexibility and avoid vendor lock-in." https://gsilvapt.me/posts/moving-to-github-actions/ https://gsilvapt.me/posts/moving-to-github-actions/ "Actions are a Vendor Lock-in Mechanism" "But are you really sparing that many lines of code? Ultimately, you’re going to have to declare what your environment needs so other developers can contribute to the project anyway, so why not simply write a bash script to take care of that for you?"
• plugin-baby 3y ago
  Local actions can be defined with a dockerfile. I haven’t used this approach extensively, but it seems like a simple way to use actions while also removing significant dependence on the platform.
• ttfkam 3y ago
  Why not just use act? It was made for this exact set of problems? https://github.com/nektos/act https://github.com/nektos/act
  • airpoint 3y ago
    Because it’s incomplete and unstable
  • TheCleric 3y ago
    I use it when I can, but in my experience it’s only useful about 50% of the time.
• dboreham 3y ago
  Yes Github actions is (are?) a mess lacking conceptual integrity and widely misused. But we do the following with success: 1. Jobs should always (just) execute a script or program. This allows running outside the CI system. 2. To test/debug CI jobs, use act. 3. To test/debug more complex scenarios spin up a Gitea instance (which provides a Github clone wrapping act).
  • mdaniel 3y ago
    > 2. To test/debug CI jobs, use act. I can't wait for this meme to die, or for act (or gitea's fork thereof) to catch up to the hype train. Then again, I guess this fantasy is being promoted by folks who are all "just use run: and that's it" because any moderately complex one <https://github.com/VSCodium/vscodium/blob/1.84.2.23314/.github/workflows/stable-linux.yml https://github.com/VSCodium/vscodium/blob/1.84.2.23314/.gith...> for sure fails with spectacularly illegible error messages under both act and gitea's fork
    • hn92726819 3y ago
      > I can't wait for this meme to die This is not helpful. You, the author, and random commenters scattered around keep teasing that act is bad, but I can't seem to find our what any of you mean. One comment said github APIs fail, which would make sense to me. Is that the primary reason for act being a pain? Do you have output for the linked build yml or an explanation of where it goes wrong?
      • mdaniel 3y ago
        I feel I'm being trolled, but I'll bite and accept the resulting downvotes I don't think treating every mention of act as an opportunity for airing of personal grievances is helpful in a discussion when there's already ample reports of people's concrete issues with it, had one looked at the 800 issues in its repo https://github.com/nektos/act/issues?q=is%3Aissue https://github.com/nektos/act/issues?q=is%3Aissue or the 239 from gitea's for https://gitea.com/gitea/act_runner/issues https://gitea.com/gitea/act_runner/issues or whatever is going on with Forgejo's fork https://code.forgejo.org/forgejo/act https://code.forgejo.org/forgejo/act . But, as for me specifically, there are two and a half answers: I wanted to run VSCodium's build locally, which act for sure puked about. Then, while trying to troubleshoot that, I thought I'd try something simpler and have it run the lint job from act's own repo <https://github.com/nektos/act/blob/1252e551b8672b1e16dc8835d6ed82c2de2c3a0c/.github/workflows/checks.yml#L11 https://github.com/nektos/act/blob/1252e551b8672b1e16dc8835d...> to rule out "you're holding it wrong" type junk. It died with [checks/lint] Failure - Main actions/setup-go@v3 [checks/lint] failed to create exec: Error response from daemon: Container b9059f831d3a1549c9902cabc7e5258231b2a7291b2692eaa33f976794059738 is not running and no amount of --verbose would tell me what, specifically, "Failure" means or what action I'm supposed to take about it Finally, I believe I am pretty self-service when projects are written in sane programming languages and publish under a permissive license, so the "and a half" is that I tried to introduce some better logging or fix up some of the more egregious silliness that I found in the codebase, but ultimately it felt like I was pushing a boulder uphill so I let that go. I haven't been following the progress of any of the 1100 forks on GH, or the hundreds of Gitea forks, or whatever the hell is going on with Forgejo nowadays But my experience is that if the thing can't run its own workflows, or at least emit some constructive action that I, the user, can take about that situation, then the VSCodium one that I linked to has no prayer
• GiorgioG 3y ago
  The over-complication of software deployments is just mind-blowing. I hope the end of cheap money reigns in the insane bloat that is DevOps today. It doesn’t have to be this way, but there continues to be too few rational adults in the room.
  • BigJono 3y ago
    It's not a zero sum game. All the enterprise companies hiring 3,000 engineers only need a tiny fraction of their software to run their core businesses, and most of it is legacy. When a tech recession hits and clears out the clown car, everyone stops paying for software. You might be working on a team of 3 experts instead of 30 morons, but you'll work twice as hard for half the pay, with the same dickhead managers that treat you as replaceable cog, except this time they'll be right. If I were you I'd stick with the easy money where you get to look like a genius for being able to write basic simple code.
  • throw555chip 3y ago
    Last year George was a football star. George was injured in a big game. George learned to code YAML+Python scripts. George is now a DevOps rock star.
• _cenw 3y ago
  The runner is open source, but once you read the code you'd wish it wasn't. https://github.com/actions/runner https://github.com/actions/runner For anyone in the audience who didn't know: GitHub Actions is based on Visual Studio Team Foundation Server's CI, and later Azure DevOps. And nobody, including the current maintainers, seem to know exactly how it all works (or how it doesn't).[1] The random exit codes is just the cherry on top! [1]: https://github.com/actions/runner/issues/510 https://github.com/actions/runner/issues/510
  • zamalek 3y ago
    I attempted to read that code in order to glean information about how it communicates back to GitHub about progress etc. The goal was to make a new runner. I ran for the hills, fast.
    • capableweb 3y ago
      Sounds like most Microsoft projects to be honest. Not a lot of them can be highlighted as the pinnacle of software engineering, including their GitHub Actions implementation.
      • zamalek 3y ago
        No kidding. I have been in the bowels of EF at work, and it has led me to (likely not independently) coin "encapsulation infatuation". Bloody hell, what a beautifully decoupled mess. The only way to make sense of a single thing is to step through/into each line of code.
    • SOLAR_FIELDS 3y ago
      It’s easy to look at the endgame. Just look at the hurdles https://github.com/nektos/act https://github.com/nektos/act has to work with
      • zamalek 3y ago
        Sadly that's an emulator of the entire stack, I wanted to do the runner part only (the goal being to end up with a sane self-hosted runner replacement).
• lucasfcosta 3y ago
  This was my first startup idea, and the first one we executed on: an open-source CI that runs on your machine exactly like it does in the cloud and doesn’t use YAML as the main configuration format. Others have tried the same, including Cicada, which was built by the same folks as Fig. Here are the failure modes for this idea: - CI succeeds most of the time. It’s only in the minority of cases that running CI locally is useful, as you don’t need to debug that often. - The only case where CI fails more often is when building pipelines. If you’re building pipelines then GitHub actions already has bazillions of actions and recipes available. Their “whole product” is a huge advantage (shout-out to readers of Crossing the Chasm). - The friction to adopt GHA is much smaller as it’s one click away on GitHub and all your other stuff is already there. - Making an MVP of CI is really difficult because it requires lots of features. You can’t even start if you don’t have things like permissions, outputs for different workflows, and a significant amount of infra for orchestration. - Even when you have an MVP the switching costs of CI are ridiculously high, and it’s a critical piece of infrastructure, so it’s difficult to get people to move over if you’re only offering marginal gains. - GitHub is owned by Microsoft, which owns Azure. Their compute will be way cheaper 99% of the time. Your product advantage must be significant enough to justify higher costs. In any case, their margin will be ridiculously higher. It’s funny how many people get to the same conclusion, yet this is a really difficult problem to solve. TL;DR: the author idea is great and many people thought about it before. Executing such an idea is extremely difficult though.
  • oooyay 3y ago
    > CI succeeds most of the time. It’s only in the minority of cases that running CI locally is useful, as you don’t need to debug that often. It's also greatly impacts migration time which would be a leading signal for any team considering a transition.
  • adobrawy 3y ago
    > GitHub is owned by Microsoft, which owns Azure. Their compute will be way cheaper 99% of the time. I hope that regulators in Europe will start to notice and respond to these monopolistic practices of IaaS players who unfairly compete on the managed services market by providing services below cost. Previously, such actions were taken against e.g. Microsoft to ensure that Windows licenses are offered on equal terms to all cloud providers (without preferences for Azure).
• rubenfiszel 3y ago
  We have built an open-source generic workflow engine to run arbitrary scripts (https://windmill.dev https://windmill.dev) with a vscode extension to build the yaml using a low-code builder and each individual script in their dedicated python/ts files so you get your full editor assistants https://youtu.be/aSOF6AzyDr8?t=116 https://youtu.be/aSOF6AzyDr8?t=116 One of the area we are expanding next is a github app so you get exactly the same UX as github actions but running windmill workflows on your windmill workers.
  • replwoacause 3y ago
    No pwsh support?
    • rubenfiszel 3y ago
      We do support pwsh :)
      • replwoacause 3y ago
        Great! Will give it a try.
    • throw555chip 3y ago
      I had to Google this, apparently pwsh is the incarnation for PowerCrap I mean PowerShell.
      • replwoacause 3y ago
        You had to Google that? It’s been widely used since 2018 to refer to PowerShell.
• semiquaver 3y ago
  > Hosting a git repo is hardly more than providing a file system and SSH access. The actual mechanism they use to keep you on their platform is the CI-pipelines Then why was GitHub so popular for the 10+ years it had no built in CI system?
  • deleted 3y ago
    [deleted]
  • IshKebab 3y ago
    Yeah, utter nonsense. The mechanisms they use to keep people on GitHub is a) network effects (easy to create issues, PRs etc because you already have an account), and b) GitHub is actually really good!
    • hotnfresh 3y ago
      But aside from enterprise SSO, far better permissions management than you get with SSH and Linux filesystem permissions, a unified open-source project discovery and vetting-assistance system, secrets management, integrated CI, lfs support, issue tracking, a billion integrations for-free, automatic dependency vulnerability detection, et c… What have the Romans done for us?
  • phkx 3y ago
    I‘d also rather say that it was about hosting the code for free in the first place and then the pull requests, including the possibility to comment on code in PRs. I use the git cli (instead of some UI / IDE extension) for all interactions with the repo locally. But as soon as it‘s about collaboration, these platforms come into play. I just quickly scanned, to find that there is the `git request-pull` command, before I wasn’t even sure whether pull requests are a git built-in feature at all. Side question: does any code hosting platform allow to comment on lines of code outside of pull requests? I‘ve had several occasions where I wanted to ask, why something was written the way it was.
    • throw555chip 3y ago
• keepamovin 3y ago
  I actually really like GitHub actions, I read the article and while I get some of the concerns, others I don't understand. In any case the author's situation doesn't apply to me, and I wanted to share something I really liked about GHA. So, I recently figured out a way to host a remote browser on them by using an Ngrok tunnel. It's really cool to see BrowserBox running from inside a GitHub action container. I literally couldn't believe it actually worked when I first figured it out! I was so excited. It started as just this tech prototype in my mind (could this be possible? Probably not but I Feel like it could be). And to see it actually achieved so cool! :) It has made CI integration testing SO much easier and more repeatable. I love that it can just run it up on Ubuntu and I can verify. Anyhow, I thought this was so cool, and such a useful way for people to either just get started with BrowserBox trying it out, or even run a quick little VPN-like/proxy browser from another region. I've even logged into HN from it on the sly. I liked this whoel concept so much, that I even wrote an action that integrates with issues to make the process as easy as possible for people. Basically you can just clone or fork the repo: https://github.com/BrowserBox/BrowserBox https://github.com/BrowserBox/BrowserBox and then open an issue and pick the template that is like "Make VPN". The login link will get published in the repo. The link is not private (unless you make your fork or template private) and there's a bit of setup with your ngrok API key (free is OK) but the issue conversation automatically guides you through all that. I thought this was so cool (free server time, actually working app), that I even created another version that uses MS Edge under the hood instead of Chrome in the original, just to show how easy it is: https://github.com/MSEdgeLord/EdgeLord https://github.com/MSEdgeLord/EdgeLord Just a niggle is that the other services we normally have (secure doc viewer, audio, remote devtools) do not work as ngrok only maps 1 port. I could use an ngrok config file I think to fix that but somehow, easy as that is, I have not gotten around to it! Another niggle is I noticed the auto-tab opening used in the GHA demo seems a little funky lately, and you may need to manually reload or resize them to un-wonkify it. Probably a little regression! Anyway! :)
  • adobrawy 3y ago
    You might also be interested in https://github.com/mxschmitt/action-tmate https://github.com/mxschmitt/action-tmate, which enables SSH to workers for debugging. If you have trouble reproducing issues on CI, this can be a life saver (or at least save a few hours of commit-push-wait rounds).
• asylteltine 3y ago
  Yaml shouldn’t be used for anything except configuration files. It’s just lazy for services to use it as a kind of DSL These kinds of systems should use an actual programming language or their own DSL
• jchw 3y ago
  A big problem with making an abstraction over CI providers is that they are, on the surface, very little more than wrappers around steps made up of small scripts, but that abstraction leaks like hell. So I think you would wind up needing to go quite high-level to make something that is useful here, but even if you do do this, the variance in runtime environment will make it very difficult to make sure things are relatively portable, and attempts to work around this will lead to problems (e.g. you could just wrap everything in Docker, but fetching large images on each CI invocation isn't really a good use of resources.) Let's say you have a project using CMake with vcpkg. Vcpkg has support for the GitHub Actions Cache API, so it can integrate very well into GHA and provide good performance for caching dependencies so they do not need to be built on each invocation. Doesn't seem too unreasonable, you could have a vcpkg+CMake plugin of some kind that tries to do the best thing for each CI provider. But, what version of vcpkg do you use? what version of CMake do you use? Vcpkg is essentially just a rolling-release Git repo, there's no specific version to use. But it would save a lot of time to make use of the vcpkg installation that already exists on GitHub Actions images. And obviously, a tool like CMake depends super heavily on the environment around it. A lot of people use the Ninja provider for better performance... but if you use it on Windows, you need to be careful: CMake with the Ninja provider on Windows can not automatically pick up the Visual Studio installation, unlike the default Windows provider (which is, well, Visual Studio.) So instead it might pick up MinGW, which is often unexpected, and does not always work correctly. Does Microsoft have an answer for how to deal with this? Not really. You might think to use the official setup-msbuild step, which has been mistakenly recommended by Microsoft, but it only sets up enough environment to do .NET development. So actually your CI abstraction needs plugins that deal with: - Every CI provider - Using every operating system - For each build step And, also, it needs to be updated fairly frequently, especially if you want to keep up to date on e.g. macOS versions. Personally, I think this is an unwinnable situation, though I'd love to be proven wrong. I think it would take something fairly dramatic to actually solve this problem most of the way, though. Something that might need a lot of collaboration between stakeholders, possibly the CI providers themselves, and probably even build/packaging tools at some level. One thing I thought was interesting was the approach of using Nix[1]. I've tried rudimentary implementations of Nix with GitHub Actions though, and it does definitely have some issues still. The most obvious issue is that it doesn't really support Windows, followed by the fact that binaries made using Nix are generally not going to be very easy to distribute (unless you're building an OCI or AppImage or something like that.) But for just running tests, it's enticing at least, and it can do a whole lot of caching in theory (for dependencies; you'll still need to handle caching of intermediate files for local tests/builds in your build system underneath Nix.) The more I think about this problem, the more I wonder which angle is the best one to come at it from. Is this ultimately a problem about software SDKs in disguise? [1]: https://determinate.systems/posts/nix-github-actions https://determinate.systems/posts/nix-github-actions
• rollcat 3y ago
  I've always been opposing push-to-deploy, unless you actually have per-branch, ephemeral test environments, and take the required care to set up your data fixtures - which is a major technical challenge on any non-trivial project. The trick is to keep the deployment code simple and stupid, while making the process transparent and dependable for the team. For production environments, rather than thinking in terms of pushing a commit, you want to reframe the problem as promoting a version from a staging environment - preferably subject to approval from QA and/or project owner. ChatOps is great for that, and it doesn't have to be painfully complicated, or full of hacky Hubot code - you can get huge ROI from a simple script[1] that posts a message at the start&end of a deployment, which includes a version string (autogenerated from a git tag[2]), environment, maybe your login/host name ("$(id -un)@$(hostname)"). CI can build the artifacts, then your job is just "./deploy prod v3.14", which is also easily delegated. [1]: def slack_post(data): req = urllib.request.Request( "https://hooks.slack.com/services/xxx/xxx/xxx", headers={"content-type": "application/json"}, data=json.dumps(data).encode("utf8"), method="POST", ) res = urllib.request.urlopen(req) assert res.getcode() in range(200, 300) return res [2]: #!/bin/sh set -eu slugify() { sed -E -e 's/[^A-Za-z0-9]/-/g' -e 's/-+/-/g' -e 's/(^-|-$)//g' } git_tag=$(git describe --tags --dirty --always) git_branch=$(git rev-parse --abbrev-ref HEAD | slugify) case "${git_branch}" in master|HEAD) echo "${git_tag}"; ;; *) echo "${git_tag}-${git_branch}"; ;; esac
  • juped 3y ago
    push to deploy can be made okay by giving it promoting-a-version semantics, with the advantage of having a deploy-history graph, but the concept as it's typically described and the way it's used in practice push people towards being not very okay at all. your approach is less likely to be misused, which often outweighs having a deploy-history graph
    • rollcat 3y ago
      The chat log is the deploy history ;) Sometimes stupid really does beat clever. Spend your tech debt tokens on the product instead!
• sghill 3y ago
  I’d like to see better centralized support generally from CI systems. Six months of steady migration and a custom DSL seems alarmingly common. CI Wishlist: * testable outside of CI * required steps based on repo characteristics (ex: security scans) * central step and workflow definitions * controlled rollouts/rollbacks to simplify migrations
• AtlasBarfed 3y ago
  Crazy thing about this and is frustratingly obvious, as I become a little older in IT, is that this is just a workflow problem. CI build pipelines aren't even really in the class of workflow problems that are even difficult to implement... workflow engines start you down a rabbit hole towards different models of computation that eventually ends up with a turing machine anyway, but I digress. It's just really kind of ridiculous that the same basic problem gets re-implemented literally thousands and thousands of times across different platforms and use cases. The fact that the current implementation is based on some Frankenstein of former visual studio code and other stuff really, isn't that surprising. Of course! Standardizing workflows leads precisely to the XkCD referenced in the article, in standards produced by the workflow vendors of of course been horrendous and focused on preserving or enhancing market share, and we're done in the era of XML. The issue of course is that workflows are an "Enterprise" feature and solution. Perhaps someday in the halcyon future someone will actually address a useful cross platform workflow model I could conserve algorithms, processes, techniques, and maybe even plugins. Because right now the workflow engine is balkanized across every single CI platform.
  • marcosdumay 3y ago
    > It's just really kind of ridiculous that the same basic problem gets re-implemented literally thousands and thousands of times across different platforms and use cases Why not? It's incredibly easy and quick to implement one, why not just make one perfectly fit for your use case every time? The actual issue here is people insisting on offshoring that implementation to GitHub, so that they deal with a brittle, badly defined interface that is not aligned with their needs, instead of just starting with a simple engine and reimplementing whatever they need. It's actually the opposite problem that standards were meant to solve.
    • baq 3y ago
      > It's incredibly easy and quick to implement one, why not just make one perfectly fit for your use case every time? Look around, it’s so easy there are corpses of companies and projects all around the industry.
• kyrofa 3y ago
  I would respectfully suggest that the author is misusing CI. If you have trouble running your tests locally, you have a problem. If you have trouble deploying from your local code, you have a problem. All of those capabilities should exist as simple scripts in your project already. Once you have that done, the CI yaml is a simple glue layer that defines an order of operations, e.g.: 1. Run static tests 2. If those pass, run unit/integration tests 3. If those pass, deploy If you find yourself screaming about YAML, you're leaning too heavily on it and need to refactor your project's scripts. Maybe a good question to ask would be "if I had to switch to another CI system today, how hard would it be?" If the answer is "hard", perhaps you're leaning too heavily on it and need to refactor your project's scripts.
  • digitalsushi 3y ago
    Testing locally, I agree with. Deployments, I think it'd be fair to consider the requirements. At work, our softare can be tested locally but deployments are all registered against a central authority, and after a point of composing enough access requirements, only then does a role (cicd in this case) have enough policy allowance to perform a deployment. The entire transaction is auditable. And I think that with a deployment, that's how it should be; allowing that trust down to a local environment strikes me that too much permission is accured with a single entity. I guess that we could better define what a deployment is; to some nonprod environments I'd agree, but I'd still probably insist on the heavy machinery up at the test, perf, qa, areas, and then getting into staging and prod, there'd be no wiggle room.
    • kyrofa 3y ago
      Fair critique, totally depends on what kind of software we're talking about and where the deployment is happening. In general though, how screwed are you if your CI environment goes down? Can you not deploy anything? That would be scary. My point, however, was mostly that the logic necessary to deploy should live as part of your codebase, not written out in YAML. The privs necessary to deploy are a separate discussion.
  • msm_ 3y ago
    This sounds fine and well, but it's not how Github Actions work (or at least, not the encouraged workflow). Let's have a look at the snippet from one of the projects I work on: - name: Set up Docker Buildx uses: docker/setup-buildx-action@v2 - name: Build and push mwdb-core image uses: docker/build-push-action@v4 with: file: ./deploy/docker/Dockerfile tags: | certpl/mwdb:${{ github.sha }} certpl/mwdb:master cache-from: | type=registry,ref=certpl/mwdb:buildcache outputs: type=docker,dest=./mwdb-image - name: Upload mwdb-core image uses: actions/upload-artifact@v3 with: name: mwdb-image path: mwdb-image Good luck running this locally. There's no script code to speak of, just references to external "actions" and parameters (for example, https://github.com/docker/setup-buildx-action https://github.com/docker/setup-buildx-action). Some CI platforms are just a simple glue layer (Gitlab CI - which I prefer - is one of them), but in most cases Github CI is not. Maybe it adds to the author frustration?
• buster 3y ago
  What? You hopefully do run your tests locally. The premise that you have to run them via the GHA configuration is only an annoyance. Same as I've never seen people running the Jenkins pipelines locally. In the end it's up to you to develop your tests "local first".
  • digitalsushi 3y ago
    After two forced CICD migrations, (shoulda been the first), I learned a pattern I think must be very common: decouple absolutely everything you can from your CICD. Small Jenkinsfiles, no groovy libraries, or Bamboo specs; Keep all of it in portable shell script or some other easily movable environment, and call it from the tiniest pinning you can in your Jenkinsfile. Then have your team use the scripts locally, and make sure that Jenkins is basically just a person on your dev team calling it the same exact way. Jenkins will end up testing your build/test/deploy solution for free and you can bring bugs it catches back to your team so that they are wasting less time on those bugs and more on product work.
    • baq 3y ago
      This is very wise and I recommend this to everyone, but is not without problems; notably if you want to parallelize the build in a stateless build system with ephemeral runners (like GHA) you have to be very careful about caching your intermediate artifacts. This can be a lot of complexity which doesn’t exist on a single build box.
• aaomidi 3y ago
  Honestly the fact that you can’t run this locally is the main problem here
• hnlmorg 3y ago
  This is why you should write your business logic in external scripts and reference them from your CI/CD. So the YAML is just an interface while your pipeline logic is CI/CD agnostic.
• 8organicbits 3y ago
  > The fact that you have to push a commit to the repo to test a change, and then wait for a runner delays everything indefinitely, and makes iterating really painful. Iterating fast is key. As mentioned `act` helps a lot, getting as much as possible to run locally really cuts cycle times. GitHub Actions feel like bash to me. They are not great languages, but they are available so often that knowing them is a net productivity boost.
  • TheCleric 3y ago
    I agree and use act as frequently I can, but it has some pretty hard limitations. For example your runners better be using Linux as act doesn’t support Windows or Mac runners.
• iandanforth 3y ago
  Other people must have very different setups to warrant this kind of additional overhead and tooling. In our repo local testing is first. You check out the code and can run the tests. The github runners do the same steps the readme encourages humans to do when running tests locally. Also I'm not sure what duplication the author is avoiding with their DSL as Github actions can be broken into components and reused and parameterized. Of course if you don't like YAML you're never going to be happy until you've wrapped it in your own layer of tech debt.
  • WirelessGigabit 3y ago
    This is my approach too. I don't use any of all-in-one actions. I write the things that I need to have locally, in bash, or a build.rs script or via package.json. Then I invoke those things in GitHub actions. I split them up so they can run concurrently.
  • benrutter 3y ago
    The times I've seen more complex testing set-ups that use actions is normally some combination of: 1. Testing involves some kind of complex environment 2. IT infrastructure in some corporations makes setting up that environment impossible. I worked once in a role where I had to write a library aimed for working with spark, but getting spark (i.e. java etc) installed on my machine involved weeks worth of requests and escalations to IT. A lot of libraries (I'm thinking of adlfs as a good example which interacts with azure data lake file systrmd) have to have relatively complex testing involving docker by nature of what they are. None if those points make "put in a PR to run tests" a good or justifiable workflow. But I understand how people wind up falling into that trap.
• sshine 3y ago
  We run CI in an image generated by our devcontainer Dockerfile. All the commands in the ‘run:’ steps of GHA can be run locally in the devcontainer. Most commands work outside in random Linux environments, but a few things will break.
• nimbius 3y ago
  patiently waiting for the "github actively considered useless" post on medium.com at some point. actions are a howling trainwreck. Copilot is a dancing bear of potential litigation and code exfiltration. neither are immediately very dangerous though as Github.com itself, and its ecosystems, have had more than sixty outages this year alone. if you can get over those, you'd better hope members of your team dont live in --or have been actively sanctioned by-- the United States because Microsoft will just delete your work and lock you out. Look...when you drop 7.5 billion dollars on a code ecosystem in order to capture the worlds developers, its incumbent upon you not to squander the braintrust youve just aligned yourself with. Microsoft feels like its done nearly everything it could to erase whatever goodwill github had as a standalone company in an effort to do the same pathetic whitewash its done with every other acquisition. Shoehorn its own garbage languages into the product, migrate its services to Azures low-performance dumpster fire, and lock in whatever users ya got in order to staunch the potential bleed to other much more meaningful programs like gitea or github. I give this github.com another 7 years before Microsoft quietly packs it up, sells it to a capital management firm, and sweeps it under the rug.
  • holman 3y ago
    I wouldn’t deny that Microsoft brings a lot of friction and problems to things within GitHub, but you also shouldn’t underestimate that GitHub itself is doing a lot of this on their own merit. Regardless of the parent company, the GitHub org is huge, with a lot of people and execs involved. Just wait until the next year or two of reimplementing all these existing features in React- that’s really going to be fun!
  • WOnderFullGOnzo 3y ago
    [dead]
• samcat116 3y ago
  This applies to just about any CI/CD program, so I guess they just don't like them in general.
• csnover 3y ago
  In addition to the suggestions others have made for locally testing workflows, there are also reverse shell actions[0] that can be used for troubleshooting CI failures on the GH runners themselves. [0] https://github.com/mxschmitt/action-tmate https://github.com/mxschmitt/action-tmate
• timcavel 3y ago
  [dead]
• eezing 3y ago
  I’ve been using a Makefile in front end projects for years- use it for all dev, build, test, and deploy scripts. Local users and GH Actions use the same Make targets. Our GH Actions are very lean.
• djha-skin 3y ago
  The idea of a DSL over YAML is also a good one. Jenkins got this right, using a DSL on top of Groovy. This allows for Jenkins build script libraries[1]. I have used them in the past, and they were a joy. They allowed for standardizing builds across many different small git repos, each with their own terraform or helm chart. I miss this feature in GHA. 1: https://www.jenkins.io/doc/book/pipeline/shared-libraries/ https://www.jenkins.io/doc/book/pipeline/shared-libraries/
  • RichardCA 3y ago
    This is a topic of debate in my current job. We use Jenkins Multibranch Pipelines and don't want to discard all that work, but we also don't want developers doing release management based on the Jenkins UI/UX. It seems logical to use GHA with local runners to call Jenkins via bash scripts running curl commands, except for how "Rube Goldberg" that sounds. But it seems like everything is like that these days, and all you get to do is pick your poison. There's also the problem of Jenkins in general, that it's a mature product and you never know if the one plugin you depend on is going to stop being maintained. Or you know you have CVE's but can't upgrade without running two environments, one production and one with all the plugins updated. Or you're chasing down an issue only to find bugs that have been open issues for several years. https://issues.jenkins.io/browse/JENKINS-52362 https://issues.jenkins.io/browse/JENKINS-52362 https://issues.jenkins.io/browse/JENKINS-52966 https://issues.jenkins.io/browse/JENKINS-52966
    • djha-skin 3y ago
      It honestly sounds like you've simply outgrown Jenkins in terms of scale. Jenkins is an excellent product but it's not very good at dealing with more than a pizza sized team's worth of CI pipelines. Once the number of teams grows past four or five, you either split them off into their own Jenkins server (which I unfortunately recommend) or you face the pain that follows.
• yesimjack 3y ago
  > The fact that you have to push a commit to the repo to test a change, and then wait for a runner delays everything indefinitely, and makes iterating really painful This isn't correct. The workflow_dispatch trigger specifically exists to allow for manual runs and you can also re-run existing failed (and successful) jobs.
• WOnderFullGOnzo 3y ago
  [dead]
• ligurio 3y ago
  Cirrus CI had an attempt to make CI-agnostic tool for running Dockerized tasks, see [1]. (I'm not affiliated wtih Cirrus CI) 1. https://cirrus-ci.org/blog/2020/10/07/cirrus-cli-ci-agnostic-tool-for-running-dockerized-tasks/ https://cirrus-ci.org/blog/2020/10/07/cirrus-cli-ci-agnostic...
• Ayesh 3y ago
  This article sounds like an incoherent rant against CI/CD in general. Coming from Jenkins, App Veyor, Circle CI, and GitLab CI, GH Actions was such a fresh breath and is a very cohesive solution to the cluttered CI space. Most of the "points" mentioned in the article are merely code smells of not using a decent deployment/testing setup. Having centralized build setups is not new either. Ubuntu PPAs and Arch's equivalent of package builds share similar pain points, but it's been serving us for decades at this point.
• yeck 3y ago
  > the runner is closed source My team runs our GHA runners on our own infrastructure using Actions Runner Controller (ARC). > The fact that you have to push a commit to the repo to test a change, and then wait for a runner delays everything indefinitely, and makes iterating really painful. Sounds like a mistake. Steps in jobs can often be run locally, and you could make choices in your jobs to make that more or less true. While there are certainly some things that you can only realistically test by triggering a CI job, many things can be tested locally. > writing pipelines in YAML is just painful... I invested 3 days to write a small DSL that compiles to GHA Yes, that can definitely be the case. But the thing about YAML is that it can easily be made a target for DSLs, as you demonstrated. I'd rather GHA use something common like YAML than force another DSL on to the world. It is better for users to decide if/what abstractions to use in front of YAML.
• jiggawatts 3y ago
  Local execution of GitHub actions for testing will become possible some time after pigs learn to fly. Everyone here is asking for it as if it’s some minor oversight, soon to be rectified. Unfortunately, this tech stack is a significant revenue source. Microsoft charges for pipeline minutes, concurrent runs, etc… This is especially true in Azure DevOps which shares much of same underlying pipeline software. Letting anyone run this locally for any reason would let them bypass the monetisation. It’s the same reason that ad-supported YouTube is “missing” a download offline feature. It’s not an oversight. It’s not happening. Stop asking. The only thing we the dev community can do about this is to develop our own open-source CD platform with blackjack and hookers. In other words, we need a Kubernetes of CI/CD.
• marcosnils 3y ago
  Hey there, Dagger employee here. We've heard this feedback from a bunch of our users, and even though the awesome work of nektos/act has helped some of them, we feel like there might be more to just try to emulate GHA locally. After all, GitHub's doesn't have any incentives to allow their users escape their platform. Having said that, our community has pushed the boundaries of Dagger and has made awesome progress on streamlining GHA into our platform. Here's two in-progress demos about some of that work Source to action: https://youtu.be/EgOvSbwKVts https://youtu.be/EgOvSbwKVts Running GHA in Dagger (project gale): https://youtu.be/rHk3spfCHFQ https://youtu.be/rHk3spfCHFQ We could use some feedback and help to make this even better. If you feel like that might be interesting, we're very active in our discord server: https://discord.com/invite/dagger-io https://discord.com/invite/dagger-io
  • mdaniel 3y ago
    The non-video TruStacks link: https://github.com/TruStacks/trustacks#trustacks-software-delivery-engine https://github.com/TruStacks/trustacks#trustacks-software-de... (Golang; GPLv3) The non-video Gale link: https://github.com/aweris/gale#github-action-local-executor https://github.com/aweris/gale#github-action-local-executor (Golang; Apache 2)
• xinayder 3y ago
  I unironically said GitHub Actions was the worse thing they made when it came to CI/CD. I tried setting up a local runner for Forgejo Actions (compatible with GHA) on my k8s cluster and it was a pain to properly set it up, 1) because Forgejo/Gitea Actions is in beta and k8s support is experimental and 2) because GHA has a very dumb syntax. The thing others mentioned: you just add an external action and you have absolutely no idea what it's doing and you barely have useful output from it. I'd rather continue using CI like Woodpecker or any other one because at least you can understand what's going on behind the scenes, but GHA is inherently bad. EDIT: author makes a point about confusing terminology from GHA: > I think some disambiguation is in order: The whole thing is advertised as "GitHub Actions", however doing something is defined in a "workflow", and workflows can be packaged, parameterized, and reused in other workflows, and this is then called an "action" again. I noticed this when I was setting up my local runner. What is an action, is Action the CI program itself or the job or a stage of the pipeline? What is the workflow, is it a job or a stage? Now I'm more convinced that this is just something stupid made by Microsoft, instead of relying on existing infra they wanted to create something "entirely new" just to force people to use GitHub. Traditional big corpo fuck you right there.
• hervem 3y ago
  Not sure if related, but recently read about https://github.com/Cicada-Software/cicada https://github.com/Cicada-Software/cicada which looks like to abstract GithubCI and GitlabCI.