5 ms·
If you want the ability to run as much of your GitHub Actions locally as possible (which you should! Testing via pushing commits is painful) you can arrange you
by simonw 3y ago
If you want the ability to run as much of your GitHub Actions locally as possible (which you should! Testing via pushing commits is painful) you can arrange your actions so the majority of the logic lives in shell scripts (or Python scripts or whatever) that are called by the workflow.
Then you can run those scripts locally and iterate in them before you try them on GitHub Actions.
I agree: Actions would be more useful if I could run the entire stack locally (an official implementation, not a community-maintained clone). But it's not a big enough inconvenience for me to care very much.
- mplewis 3y agoIncidentally, this is sort of how the Earthly runner serves to provide you local actions that also work in CI.
- simcop2387 3y agoIsn't that exactly what the self hosted runners are? https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners https://docs.github.com/en/actions/hosting-your-own-runners/... Then you configure the repo settings to only use the self hosted ones.
- jahsome 3y agoIf you're suggesting the self-hosted runners are equivalent to running locally, then no, self-hosted runners still require a commit + push and runs remotely with the added overhead of the job being queued, reported, etc. The dream is to have something like a container _locally_ and be able to run something like: > gh actions run
- dinkleberg 3y agoThis is the way. It also lets you pretty much seamlessly switch CI tools at any time.
- jahnu 3y agoI third this opinion: another reason it’s important is it allows you to do your build in an emergency when CI is down for whatever reason. CI automation is for me the thing that replaces me running scripts one by one (and reporting/deploying results) It is not the thing that does the building/testing/deploying. That’s the scripts that are hopefully written in a debuggable, portable, language.
- simonw 3y agoThis is a really good point. Most of my deploys go via GitHub Actions these days and it's not easy for me to deploy manually should I need to when Actions is unavailable - I should fix that!
- jahnu 3y agoLike aviation, every regulation/rule is written in blood ;)
- leetrout 3y agoThis is why the three of us should record a podcast episode talking about this in the context of the points made in the Ironies of Automation https://en.wikipedia.org/wiki/Ironies_of_Automation https://en.wikipedia.org/wiki/Ironies_of_Automation
- jamesfinlayson 3y agoAgreed - I had to switch from TeamCity to GitLab earlier this year and I was very glad that all of pipelines were single-line calls to shell scripts.
- jasonm23 3y agoIndeed, to rely on third party actions in a closed runner ecosystem, is simply unwise.
- notnmeyer 3y agoyes! what we did was use a task runner to encapsulate the work in a “ci” task. you can run it locally, and then the GHA runner does little more than clone, install a couple depa and run the task.
- remram 3y agoThe problem is that GitHub Action offers a large ecosystem of reusable actions that you can't easily call from a script.
- xeromal 3y agoYou, you sacrifice that convenience for something purely in shell that is more portable.
- remram 3y agoyou you?
- xeromal 3y agoIt's like a yo-yo but faster
- philipswood 3y agoLike yo-yo, but slower due to the extra W.
- simonw 3y agoI mainly ignore those - I stick to the actions that install things (setup-python) and run scripts.
- deleted 3y ago[deleted]
- maccard 3y agoDoing this also means that your build scripts are version controlled. You can iterate on the build scripts in different branches, for example.
- twic 3y agoGitHub actions are configured by version-controlled files too though. I have an actions-dev branch on my project, so I can iterate on actions config without having to clutter up master. Would still be better if I could run locally to iterate. But the local run would have to have very high fidelity to what happens on GitHub to be useful.
- philihp 3y agoRunning actions triggered from a non-main branch does it for me, either on pushes to the branch, or 'workflow_dispatch'. You'll otherwise never get that level of fidelity to what happens in production. Keep the logic in your YAML "dumb". Avoid variables and subroutines in the file, if you want to DRY something, create your own custom action. You can have unit tests on an action, you can't have unit tests on a GHA workflow.
- mdaniel 3y agoThat cuts both ways, in my experience: a bug fix needs a metric boatload of cherry-pick, possibly across a bunch of repos, to fix any bugs. Centralization is great until it's not We have gotten a lot of good mileage out of GitLab's include: feature <https://docs.gitlab.com/ee/ci/yaml/#include https://docs.gitlab.com/ee/ci/yaml/#include>, which for clarity absolutely carries this same hazard, but can mean centralized fixes are less onoz than trying to patch every active branch of every active repo's build scripts I am also aware this cuts against the thrust of the thread about "build scripts should be locally executable," but until they (they: GitHub, they: GitLab) straighten out their runner binary to be sane, it is currently a better use of our engineering effort maintaining CI yaml than teaching devs how to run CI locally
- SethMLarson 3y agoThis is the way.
- marcosdumay 3y agoBuilding Java and C# on the command line nowadays is a hard undertaking, requiring weeks of learning and debugging, just to have your code change in a way that makes the scripts outdated at any time (oh, and it's never portable). I really don't get why people use those complex CI/CD tools for other languages, but at least on the enterprise ones, I can understand people moving from the IDE into some huge centralized mess that can import it.
- neonsunset 3y agoBuilding C# is quite literally 'dotnet build -c release'.
- hypeatei 3y agoExactly. I don't know where the newer .NET versions get the bad rap from. You can criticize Microsoft for other things but .NET is not one of them. They've done a great job with it.
- Kwpolska 3y ago(And building code that targets the classic .NET Framework can usually be accomplished by a single call to msbuild.exe.)
- twic 3y agoWhy does Java take weeks? For a straightforward application or library project, you can fill in this form and get a shovel-ready build: https://gradle-initializr.cleverapps.io/ https://gradle-initializr.cleverapps.io/ I usually tweak it a bit, but that's minutes, not weeks. There's often more stuff to add as a project grows, but again, minutes to hours for the common stuff. Setting up publishing to a Maven repository is more work than I'd like, but it's still not weeks. So what are you doing that it takes so long? And how long would that take if done any other way?
- Timon3 3y agoI wish I didn't have to circumvent the UI with a strategy like this. Part of the reason I like CI/CD tools is the visualization factor - I can create specific steps, see which step is currently active during execution, and only care for the output of a specific step during debugging. A platform with support for visual control from the scripts (implemented as no-ops during local execution) would be perfect.
- truculent 3y agoIf you split your scripts into small tasks (i.e. one task per script), and call each one from your GA yaml, you would seem to have the best of both?
- Timon3 3y agoThat still makes running it a bit annoying, since you have to call each script in order. You could of course create a script that covers this, but that's duplication that can drift apart. I guess you could generate the script from your YAML or use a linter or something, but it all feels a bit annoying. It would be awesome to e.g. just call a command like "gh-success" or "gh-failure" to manually indicate the success/failure, maybe with an optional ID to prevent accidental double calls etc.
- snnn 3y agoI'm not quite familiar with GitHub Actions, but Azure DevOps Pipeline has a nice Preview API: https://learn.microsoft.com/en-us/rest/api/azure/devops/pipelines/preview/preview?view=azure-devops-rest-7.1 https://learn.microsoft.com/en-us/rest/api/azure/devops/pipe... which runs the preprocessing steps(like running the C preprocessor on a C source file) then gives you the processed yaml file. Then if you'd like, you can write your own local runner based on that if your yaml files do not use too many different kinds Azure DevOps pipeline steps.