9 ms·

GitHub Actions by Example

436 points by macintoshpie 5y ago

keewee7 5y ago
>Inspired by Go by Example One of the best tutorials for people coming to Go from other languages: https://gobyexample.com/ https://gobyexample.com/
- synergy20 5y ago
  I wish the UI has a left sidebar with scroll-able TOC, so I don't need jump back to home page to pick a different subject. I asked the same to gobyexample.com's author and was told they have no intention to change.
  - systemvoltage 5y ago
    I disagree, I think the simplicity of this page is its strength and its beauty. I wish more sites were designed this way.
    - samhw 5y ago
      I think this exchange sums up the debate over Go itself...
gfunk911 5y ago
I might love this. Last time I checked on the Actions docs, they seemed to say a lot, while still leaving me confused somehow.
- jonny_eh 5y ago
  > they seemed to say a lot, while still leaving me confused somehow Saying a lot and causing confusion usually go hand-in-hand.
  - nefitty 5y ago
    Actions was my first experience with YAML and it made me want to bark at my monitor. No, I'm not a dog.
    - phalangion 5y ago
      We can’t know that https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_you're_a_dog https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_...
    - naikrovek 5y ago
      JSON is valid YAML.
    - thunkshift1 5y ago
      How to learn the YAML syntax? Its frustrating.. are there any autocomplete tools
- mrbuzzinfrog 5y ago
  It's pretty terrible if you compare it to CircleCI or GitLab from 4 years ago. I'm a big fan of GitLab, seems like the only company pushing things forward in an _elegant matter_, used it heavily in the startup world. Using GitHub again these days. I cry every time I need to do GHA stuff. Current setup of Github + CircleCI is miles more elegant as it was 6 years ago.
  - yjftsjthsd-h 5y ago
    I like gitlab, but I'm not sure elegant is the word I'd use for them; they have a severe case of wanting to check all the boxes for features, but it can be a little clunky to see how the parts work together. My pick for elegance would be sourcehut. On the other hand, they all seem to work pretty decently and the clunkiness isn't that bad, so I keep using it:)
  - jpthurman 5y ago
    This makes sense - the main reason GitLab took off is vertical integration with CI/CD which Github is catching up on. Github has the long game in mind and with it's size and preponderance of OSS I see it taking over when they innovate to a more useable level.
    - naikrovek 5y ago
      well, look at Azure DevOps. Everything it does is coming to GitHub, and AZDO will eventually be sunset in favor of GitHub. And AZDO is quite good, imo. MS moved a lot of (almost all) Azure DevOps people and put them on GitHub.
pqb 5y ago
Nice idea, worth mentioning other features: - Reusable workflows (note: matrix strategy doesn't work here): https://docs.github.com/en/actions/using-workflows/reusing-workflows https://docs.github.com/en/actions/using-workflows/reusing-w... - Creating an action: https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions https://docs.github.com/en/actions/creating-actions/metadata... - Composite actions: https://docs.github.com/en/actions/creating-actions/creating-a-composite-action https://docs.github.com/en/actions/creating-actions/creating... - Fact, the "uses" in step can be a relative path in the same repository (obviously, you must checkout the code, when it uses a relative path): https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-using-versioned-actions https://docs.github.com/en/actions/using-workflows/workflow-... - Script as an action: https://github.com/actions/github-script https://github.com/actions/github-script - Using GitHub Packages and artifacts: https://docs.github.com/en/actions/publishing-packages/about-packaging-with-github-actions https://docs.github.com/en/actions/publishing-packages/about... - Using docker-compose-like services that run alongside of the container: https://docs.github.com/en/actions/using-containerized-services/creating-postgresql-service-containers https://docs.github.com/en/actions/using-containerized-servi... - Using heredoc to share multi-line JSON in an environment variable, also using fromJSON/toJSON functions: - name: get-version id: get-version run: |- echo 'JSON_RESPONSE<<EOF' >> $GITHUB_ENV cat package.json >> $GITHUB_ENV echo 'EOF' >> $GITHUB_ENV - name: print-version run: echo "${{ fromJSON(env.JSON_RESPONSE).version }}" - The matrix/strategy of the dependent workflow can be created dynamically, like in the following workflow: https://github.com/googleapis/google-cloud-go/blob/83bbc2e7c67a9452adb7a1d56ea2f58bfc6b851c/.github/workflows/release-submodule.yaml#L36 https://github.com/googleapis/google-cloud-go/blob/83bbc2e7c... And many, many more :)
- macintoshpie 5y ago
  Sick! I've been meaning to add more to this for a while, thanks for the suggestions
bilalq 5y ago
I wish more people wrote docs in this fashion. I almost always end up skipping official docs in favor of digging around for blog posts going over code examples on things.
- samhw 5y ago
  It's not unheard of. A few examples come to mind: - Go By Example: https://gobyexample.com/ https://gobyexample.com/ - Rust By Example: https://doc.rust-lang.org/rust-by-example/ https://doc.rust-lang.org/rust-by-example/ - V [a weird knockoff of Go] By Example: https://v-community.gitbook.io/v-by-example/ https://v-community.gitbook.io/v-by-example/ There's also 'Learn X in Y Minutes' (https://learnxinyminutes.com/ https://learnxinyminutes.com/), which covers a range of different 'X'es. They make it ridiculously easy to get going with a new tool/language, IMO. It's a superb paradigm in general.
- suyash 5y ago
  I loved the simplicity and directness of the UI too. Specially like the feature where mouse cursor maps to code blocks, how did you create this ? I'd like to use this framework if possible for tech blogs.
  - macintoshpie 5y ago
    Thanks! The page itself is just simple HTML and tables. I generated it from YAML files with a custom HTML generator, see for example: https://github.com/macintoshpie/ghactionsbyexample/blob/df6ff876a9ce246cfedb86f8b17a8d3461f1bf18/examples/hello-world/hello-world.yml#L1 https://github.com/macintoshpie/ghactionsbyexample/blob/df6f...
asciiii 5y ago
I really like the format and layout of this.
fierro 5y ago
the most bizarre thing about GH actions is the I/O mechanic where you produce outputs by executing `echo ::set-output name=<name>::<value>.`
- donatj 5y ago
  Agreed, the whole inline-signaling aspect of that makes me a little uncomfortable.
- tailspin2019 5y ago
  This sort of thing is pretty common in many CI tools, so I wouldn’t call it bizarre exactly. Or unique to GH. I agree that it’s a bit inelegant though.
erwincoumans 5y ago
Nice and simple explanation. Never looked at the docs, I used the Github 'workflow' to automatically create an action for CMake and CTest (C++ code) and just followed the steps without customizing. It just worked out-of-the-box, the autogenerated yaml file is here: https://github.com/google-research/tiny-differentiable-simulator/tree/master/.github/workflows https://github.com/google-research/tiny-differentiable-simul...
cstuder 5y ago
Maybe this thread is the right place to ask a usage question: I have a multiline GitHub Secret which I would like to print out to a `.env` file in a GitHub Action. How can I do that? My current solution doesn't support spaces within the secret content: - name: Write .env run: | echo $ENV_FILE | tr ' ' '\n' > .env shell: bash env: ENV_FILE: ${{secrets.DOTENV}} Writing a multiline secret string directly into a file replaces all newlines with spaces. The `tr` command converts them back to newlines.
- nagisa 5y ago
  `echo "$ENV_FILE"`, maybe? `echo $ENV_FILE` (without quotes) will split the environment variable by separators in $IFS and pass each chunk as separate argument.
- xyzzy_plugh 5y ago
  You have to quote your variable expansion. Also using echo is bad form, instead try cat <<<$ENV_FILE >.env
staticassertion 5y ago
https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions https://docs.github.com/en/actions/security-guides/security-... Probably worth checking out this guide. GHA can be a pretty scary thing.
- jshier 5y ago
  None of that is scary. Pretty much all of that advice applies to systems you run internally. I do wish GHA had a solution for secure file injection, which solutions like Jenkins already have, so we didn't need a janky workaround for JSON blobs.
nyanpasu64 5y ago
> Actions reduce workflow steps by providing reusabe[sic] “code” for common tasks. To run an action, you include the uses keyword pointing to a GitHub repo with the pattern {owner}/{repo}@{ref} or {owner}/{repo}/{path}@{ref} if it’s in a subdirectory. A ref can be a branch, tag, or SHA. Aside from the typo, I wonder how many packages could be backdoored at once, if an action maintainer went rogue, seeing as there's no pinning for actions by default, and (according to https://github.com/msys2/setup-msys2/blob/main/HACKING.md https://github.com/msys2/setup-msys2/blob/main/HACKING.md) moving a tag is the default way to push updates to an action. (Interestingly get-cmake/run-cmake/run-vcpkg are all operated by the same person.)
- macintoshpie 5y ago
  Oops thanks for the catch
- patcon 5y ago
  Def a real concern. If anyone is interested to mitigate it yourself, these are helpful :) https://docs.github.com/en/actions/creating-actions/about-custom-actions#using-a-commits-sha-for-release-management https://docs.github.com/en/actions/creating-actions/about-cu... https://github.com/dependabot/dependabot-core/issues/2835 https://github.com/dependabot/dependabot-core/issues/2835 https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions https://github.com/zgosalvez/github-actions-ensure-sha-pinne... https://github.com/timmeinerzhagen/dependabot-sha-comment-action https://github.com/timmeinerzhagen/dependabot-sha-comment-ac...
- bewuethr 5y ago
  You can pin to a commit SHA, and that's what the docs recommend as the "safest for stability and security": https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsuses https://docs.github.com/en/actions/using-workflows/workflow-...
natrys 5y ago
While dealing with an unfamiliar project recently, being able to run actions locally with act[1] was a great time saver for me: [1] https://github.com/nektos/act https://github.com/nektos/act
- terhechte 5y ago
  Thanks! I searched for something like this recently and couldn't find it! Should be linked in the Actions docs.
- thinkafterbef 5y ago
  Incoming shameless plug; if you don’t have to handle the hosting runners, but still to reap the benefits of having proper hardware (close to the metal). Check out BuildJet for GitHub actions[1] - 2x the speed for half the price. Easy to install and easy to revert. [1] https://buildjet.com/for-github-actions https://buildjet.com/for-github-actions
  - Grimburger 5y ago
- naikrovek 5y ago
  I wrote a very small container orchestrator that spawns a runner container when it gets a webhook that a job was triggered. it spawns a runner container with the environment required to attach to the repo as an ephemeral runner, do the job, then detach the runner and exit the container. very fast. this was all before I knew about act and it's about the same speed in my limited testing.
- ebingdom 5y ago
  If you're looking for a way to reproduce your CI locally that isn't tied to a particular CI system (but which has a nice integration with GitHub Actions), there's also Toast: https://github.com/stepchowfun/toast https://github.com/stepchowfun/toast Toast lets you use whatever base image you want (even when running with GitHub Actions), and it has some extra features for local development (e.g., caching, bind mounts, tasks with dependencies between them, etc.).
0xbadcafebee 5y ago
Every CI system in existence is reinventing the exact same wheel: "I want to run some random task" + event hooks + secrets + logs + plugins + integrations. It's so ridiculous that more of them keep being created - and are losing functionality. GHA has all these configs in YAML (there's a user-friendly config file...) but doesn't let you run paramaterized builds in their web UI? .....why? We shouldn't be writing all these jobs in a format that only works for one CI system. You spend months writing Jenkinsfiles, and then you move to CircleCI and have to rewrite all of them, and then move to Drone and have to rewrite all of them, and then move to CodeBuild/CodePipeline and have to rewrite all of them, and then move to GitHub Actions and have to rewrite all of them. Eventually we'll rewrite them all for something else. And why? To run the same exact tests on a slightly different system.
- yebyen 5y ago
  > doesn't let you run paramaterized builds in their web UI? It doesn't? https://github.blog/changelog/2021-11-10-github-actions-input-types-for-manual-workflows/ https://github.blog/changelog/2021-11-10-github-actions-inpu... Maybe it didn't 6 months ago, but it seems this is an option now, and well-documented. This is a topic of interest for me, I presented on Jenkins and GitOps to an audience at KubeCon who I suspect are almost all interested in moving away from Jenkins, or have been told to be interested in switching from Jenkins to something else, and I tried to get the idea across that they probably don't really need to switch the workflow tool even if it's ancient, ... But maybe should consider subbing out some of the important fiddly bits underneath it (like, I assume the vast majority of Jenkins users are building images with Docker, and if they're running on Kubernetes, they're many of them wondering what they will do, or how long they really have before they have to start worrying about the deprecation of dockershim and how their lives are going to change when their clusters won't be running Docker under the hood anymore?) I was actually arguing for a tool like Porter.sh to come in and make the boundaries of "what's in a build" super neat and tidy, organized, but also limited so the next time they feel compelled to switch workflow tooling, it will be a non-issue and can be over and finished inside of a single day's work. The problem isn't that your workflow tool is too old, it's that you've jammed too much arbitrary complexity inside of it, probably because the right abstraction was not made available to you at the time. "Switching off of Jenkins" just means building a second system and it comes with all the baggage of "second system syndrome" to do so. Sure it's difficult switching from Jenkins, when you've built this gigantic pipeline with 18 branches and 12 of them run in parallel, half of them are configured with different options passed to the docker build tool, half of them must use buildx, and the third half of them are unmaintained so we don't go in there... so put some guard rails up around the hard parts! And get somebody in there to take care of those cobwebs.
ghotli 5y ago
I too love "Go by Example" and refer to it often. Makes me want it for all the things. Shot in the dark, anyone know of one for hot-off-the-presses modern Python (3.10) with typing akin to Golang? All the modern additions really need a comprehensive overview like Go by Example somehow manages to do in a very lightweight style
Kreotiko 5y ago
Word of advice, if you are thinking of running self hosted runners and use Actions for your organisation, do yourself a favour and check them out in a year or two and use something like Argo Workflows or Tekton instead. GHA isn’t a product thought for GH private organisations, you will find that every much needed feature for this use is very low in GH roadmap.
- longcommonname 5y ago
  Could you provide more details?
  - Kreotiko 5y ago
    Sure, the main things for me are: It doesn’t matter how smart you are with reusable workflows you will never get to a truly DRY setup that scales for dozens of repositories. Another major pain is that we still haven’t private actions. It was due end of 2021 (maybe it is out now but I checked a couple of days ago). Setting up runners to look after a pool of repositories needs elevated permissions. GH offers a way to enforce a list of enabled actions but this does not work with private binary registries hosting pre built Docker actions. The only thing that could prevent you to pull software at runtime from the internet, which means, if you want to have a decent security posture all you are left with is referencing actions using the full git sha version. Many common use case require hacks, which is fun for a weekend project but isn’t great for a large scale operation. An example is simply running a workflow dynamically targeting the folders containing changes. At the moment you have to create a job, generate a build matrix on the fly and pass it in input as the matrix to the actual job.
    - sverhagen 5y ago
      Don't you have access to actions in private repositories, or is that not the same?
      - bewuethr 5y ago
        The only way I'm aware of to use a private action is to clone the repository it's in using a personal access token, and then use a local relative path to the action to run it.
filleokus 5y ago
GitHub Actions with act to test stuff locally is actually pretty usable. We use on-prem k8s-hosted runners to get access to servers/clusters with limited internet access, works great. But as always with “”gitops”” themed tools I think it’s pretty awkward to handle rollbacks. Either you take the stance that master is the source of truth and let the CI/CD tooling revert commits if deployment fails, or you store that state elsewhere and allow e.g manifests to diverge from git. I would be curious to hear how other people do it
- bilalq 5y ago
  Like all terms in the industry, "gitops" gets defined a little differently by everyone. In our case, we use trunk-based development where "main" is the branch that is approved for release, but it does not necessarily point to what is currently released. Instead, we use git tags for that. On merge to main, we kick off pipeline executions (we use AWS CodePipeline here, but Github actions with a concurrency group id set would achieve something similar). Non-prod stages just push lightweight tags in the format of `$STAGE-YYYY.MM.DD.hhmm` on successful deployment. Prod stages are similar, but we publish a Github release declaration using the Github API rather than just pushing a git tag. In the case of rollback, you either push new tags at older commits or roll forward with a revert commit (but still using tags as the tracking mechanism).
  - ynouri 5y ago
    Thanks for sharing! We are a small company and right now we release (from our monorepo) on every PR merge to main. As the engineering team continues growing, we anticipate this approach will become unsustainable soon, and are looking for alternatives. I wonder if the tag based release is a widespread industry pattern (especially for growing monorepos)? Would be curious to learn more about it
    - DrewRWx 5y ago
      I'm on a team that implemented a hybrid tag-based release system on a monorepo and it is working well. External releases are built on release branches off of main with their own git tag rules, but I want to touch on internal releases off of main. Between feature branches and main we have GitHub status checks that gate bad PRs from getting in. Once the PRs are in main, we build a nightly full suite of components and put them through various levels of interop testing. Once our system receives the signal that all builds and interop testing had passed, it applies a Last Known Good (LKG) tag to the git commit the components were built from. After that, various systems including artifact links for QA and auto-merge jobs from main, are set to use the LKG tag to ensure they are getting good builds and code.
baby 5y ago
This is great! As feedback I would try to really enforce best practices. For example, every steps should have a descriptive name (not just “uses”)
devops000 5y ago
Cool. This should be added to the offical documentation.

436 points by macintoshpie 5y ago

>Inspired by Go by Example One of the best tutorials for people coming to Go from other languages: https://gobyexample.com/ https://gobyexample.com/

synergy20 5y ago
I wish the UI has a left sidebar with scroll-able TOC, so I don't need jump back to home page to pick a different subject. I asked the same to gobyexample.com's author and was told they have no intention to change.
- systemvoltage 5y ago
  I disagree, I think the simplicity of this page is its strength and its beauty. I wish more sites were designed this way.
  - samhw 5y ago
    I think this exchange sums up the debate over Go itself...

gfunk911 5y ago

I might love this. Last time I checked on the Actions docs, they seemed to say a lot, while still leaving me confused somehow.

jonny_eh 5y ago
> they seemed to say a lot, while still leaving me confused somehow Saying a lot and causing confusion usually go hand-in-hand.
- nefitty 5y ago
  Actions was my first experience with YAML and it made me want to bark at my monitor. No, I'm not a dog.
  - phalangion 5y ago
    We can’t know that https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_you're_a_dog https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_...
  - naikrovek 5y ago
    JSON is valid YAML.
  - thunkshift1 5y ago
    How to learn the YAML syntax? Its frustrating.. are there any autocomplete tools
mrbuzzinfrog 5y ago
It's pretty terrible if you compare it to CircleCI or GitLab from 4 years ago. I'm a big fan of GitLab, seems like the only company pushing things forward in an _elegant matter_, used it heavily in the startup world. Using GitHub again these days. I cry every time I need to do GHA stuff. Current setup of Github + CircleCI is miles more elegant as it was 6 years ago.
- yjftsjthsd-h 5y ago
  I like gitlab, but I'm not sure elegant is the word I'd use for them; they have a severe case of wanting to check all the boxes for features, but it can be a little clunky to see how the parts work together. My pick for elegance would be sourcehut. On the other hand, they all seem to work pretty decently and the clunkiness isn't that bad, so I keep using it:)
- jpthurman 5y ago
  This makes sense - the main reason GitLab took off is vertical integration with CI/CD which Github is catching up on. Github has the long game in mind and with it's size and preponderance of OSS I see it taking over when they innovate to a more useable level.
  - naikrovek 5y ago
    well, look at Azure DevOps. Everything it does is coming to GitHub, and AZDO will eventually be sunset in favor of GitHub. And AZDO is quite good, imo. MS moved a lot of (almost all) Azure DevOps people and put them on GitHub.

pqb 5y ago

Nice idea, worth mentioning other features: - Reusable workflows (note: matrix strategy doesn't work here): https://docs.github.com/en/actions/using-workflows/reusing-workflows https://docs.github.com/en/actions/using-workflows/reusing-w... - Creating an action: https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions https://docs.github.com/en/actions/creating-actions/metadata... - Composite actions: https://docs.github.com/en/actions/creating-actions/creating-a-composite-action https://docs.github.com/en/actions/creating-actions/creating... - Fact, the "uses" in step can be a relative path in the same repository (obviously, you must checkout the code, when it uses a relative path): https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-using-versioned-actions https://docs.github.com/en/actions/using-workflows/workflow-... - Script as an action: https://github.com/actions/github-script https://github.com/actions/github-script - Using GitHub Packages and artifacts: https://docs.github.com/en/actions/publishing-packages/about-packaging-with-github-actions https://docs.github.com/en/actions/publishing-packages/about... - Using docker-compose-like services that run alongside of the container: https://docs.github.com/en/actions/using-containerized-services/creating-postgresql-service-containers https://docs.github.com/en/actions/using-containerized-servi... - Using heredoc to share multi-line JSON in an environment variable, also using fromJSON/toJSON functions: - name: get-version id: get-version run: |- echo 'JSON_RESPONSE<<EOF' >> $GITHUB_ENV cat package.json >> $GITHUB_ENV echo 'EOF' >> $GITHUB_ENV - name: print-version run: echo "${{ fromJSON(env.JSON_RESPONSE).version }}" - The matrix/strategy of the dependent workflow can be created dynamically, like in the following workflow: https://github.com/googleapis/google-cloud-go/blob/83bbc2e7c67a9452adb7a1d56ea2f58bfc6b851c/.github/workflows/release-submodule.yaml#L36 https://github.com/googleapis/google-cloud-go/blob/83bbc2e7c... And many, many more :)

macintoshpie 5y ago
Sick! I've been meaning to add more to this for a while, thanks for the suggestions

bilalq 5y ago

I wish more people wrote docs in this fashion. I almost always end up skipping official docs in favor of digging around for blog posts going over code examples on things.

samhw 5y ago
It's not unheard of. A few examples come to mind: - Go By Example: https://gobyexample.com/ https://gobyexample.com/ - Rust By Example: https://doc.rust-lang.org/rust-by-example/ https://doc.rust-lang.org/rust-by-example/ - V [a weird knockoff of Go] By Example: https://v-community.gitbook.io/v-by-example/ https://v-community.gitbook.io/v-by-example/ There's also 'Learn X in Y Minutes' (https://learnxinyminutes.com/ https://learnxinyminutes.com/), which covers a range of different 'X'es. They make it ridiculously easy to get going with a new tool/language, IMO. It's a superb paradigm in general.
suyash 5y ago
I loved the simplicity and directness of the UI too. Specially like the feature where mouse cursor maps to code blocks, how did you create this ? I'd like to use this framework if possible for tech blogs.
- macintoshpie 5y ago
  Thanks! The page itself is just simple HTML and tables. I generated it from YAML files with a custom HTML generator, see for example: https://github.com/macintoshpie/ghactionsbyexample/blob/df6ff876a9ce246cfedb86f8b17a8d3461f1bf18/examples/hello-world/hello-world.yml#L1 https://github.com/macintoshpie/ghactionsbyexample/blob/df6f...

asciiii 5y ago

I really like the format and layout of this.

fierro 5y ago

the most bizarre thing about GH actions is the I/O mechanic where you produce outputs by executing `echo ::set-output name=<name>::<value>.`

donatj 5y ago
Agreed, the whole inline-signaling aspect of that makes me a little uncomfortable.
tailspin2019 5y ago
This sort of thing is pretty common in many CI tools, so I wouldn’t call it bizarre exactly. Or unique to GH. I agree that it’s a bit inelegant though.

erwincoumans 5y ago

Nice and simple explanation. Never looked at the docs, I used the Github 'workflow' to automatically create an action for CMake and CTest (C++ code) and just followed the steps without customizing. It just worked out-of-the-box, the autogenerated yaml file is here: https://github.com/google-research/tiny-differentiable-simulator/tree/master/.github/workflows https://github.com/google-research/tiny-differentiable-simul...

cstuder 5y ago

Maybe this thread is the right place to ask a usage question: I have a multiline GitHub Secret which I would like to print out to a `.env` file in a GitHub Action. How can I do that? My current solution doesn't support spaces within the secret content: - name: Write .env run: | echo $ENV_FILE | tr ' ' '\n' > .env shell: bash env: ENV_FILE: ${{secrets.DOTENV}} Writing a multiline secret string directly into a file replaces all newlines with spaces. The `tr` command converts them back to newlines.

nagisa 5y ago
`echo "$ENV_FILE"`, maybe? `echo $ENV_FILE` (without quotes) will split the environment variable by separators in $IFS and pass each chunk as separate argument.
xyzzy_plugh 5y ago
You have to quote your variable expansion. Also using echo is bad form, instead try cat <<<$ENV_FILE >.env

staticassertion 5y ago

https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions https://docs.github.com/en/actions/security-guides/security-... Probably worth checking out this guide. GHA can be a pretty scary thing.

jshier 5y ago
None of that is scary. Pretty much all of that advice applies to systems you run internally. I do wish GHA had a solution for secure file injection, which solutions like Jenkins already have, so we didn't need a janky workaround for JSON blobs.

nyanpasu64 5y ago

> Actions reduce workflow steps by providing reusabe[sic] “code” for common tasks. To run an action, you include the uses keyword pointing to a GitHub repo with the pattern {owner}/{repo}@{ref} or {owner}/{repo}/{path}@{ref} if it’s in a subdirectory. A ref can be a branch, tag, or SHA. Aside from the typo, I wonder how many packages could be backdoored at once, if an action maintainer went rogue, seeing as there's no pinning for actions by default, and (according to https://github.com/msys2/setup-msys2/blob/main/HACKING.md https://github.com/msys2/setup-msys2/blob/main/HACKING.md) moving a tag is the default way to push updates to an action. (Interestingly get-cmake/run-cmake/run-vcpkg are all operated by the same person.)

macintoshpie 5y ago
Oops thanks for the catch
patcon 5y ago
Def a real concern. If anyone is interested to mitigate it yourself, these are helpful :) https://docs.github.com/en/actions/creating-actions/about-custom-actions#using-a-commits-sha-for-release-management https://docs.github.com/en/actions/creating-actions/about-cu... https://github.com/dependabot/dependabot-core/issues/2835 https://github.com/dependabot/dependabot-core/issues/2835 https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions https://github.com/zgosalvez/github-actions-ensure-sha-pinne... https://github.com/timmeinerzhagen/dependabot-sha-comment-action https://github.com/timmeinerzhagen/dependabot-sha-comment-ac...
bewuethr 5y ago
You can pin to a commit SHA, and that's what the docs recommend as the "safest for stability and security": https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsuses https://docs.github.com/en/actions/using-workflows/workflow-...

natrys 5y ago

While dealing with an unfamiliar project recently, being able to run actions locally with act[1] was a great time saver for me: [1] https://github.com/nektos/act https://github.com/nektos/act

terhechte 5y ago
Thanks! I searched for something like this recently and couldn't find it! Should be linked in the Actions docs.
thinkafterbef 5y ago
Incoming shameless plug; if you don’t have to handle the hosting runners, but still to reap the benefits of having proper hardware (close to the metal). Check out BuildJet for GitHub actions[1] - 2x the speed for half the price. Easy to install and easy to revert. [1] https://buildjet.com/for-github-actions https://buildjet.com/for-github-actions
- Grimburger 5y ago
naikrovek 5y ago
I wrote a very small container orchestrator that spawns a runner container when it gets a webhook that a job was triggered. it spawns a runner container with the environment required to attach to the repo as an ephemeral runner, do the job, then detach the runner and exit the container. very fast. this was all before I knew about act and it's about the same speed in my limited testing.
ebingdom 5y ago
If you're looking for a way to reproduce your CI locally that isn't tied to a particular CI system (but which has a nice integration with GitHub Actions), there's also Toast: https://github.com/stepchowfun/toast https://github.com/stepchowfun/toast Toast lets you use whatever base image you want (even when running with GitHub Actions), and it has some extra features for local development (e.g., caching, bind mounts, tasks with dependencies between them, etc.).

0xbadcafebee 5y ago

Every CI system in existence is reinventing the exact same wheel: "I want to run some random task" + event hooks + secrets + logs + plugins + integrations. It's so ridiculous that more of them keep being created - and are losing functionality. GHA has all these configs in YAML (there's a user-friendly config file...) but doesn't let you run paramaterized builds in their web UI? .....why? We shouldn't be writing all these jobs in a format that only works for one CI system. You spend months writing Jenkinsfiles, and then you move to CircleCI and have to rewrite all of them, and then move to Drone and have to rewrite all of them, and then move to CodeBuild/CodePipeline and have to rewrite all of them, and then move to GitHub Actions and have to rewrite all of them. Eventually we'll rewrite them all for something else. And why? To run the same exact tests on a slightly different system.

yebyen 5y ago
> doesn't let you run paramaterized builds in their web UI? It doesn't? https://github.blog/changelog/2021-11-10-github-actions-input-types-for-manual-workflows/ https://github.blog/changelog/2021-11-10-github-actions-inpu... Maybe it didn't 6 months ago, but it seems this is an option now, and well-documented. This is a topic of interest for me, I presented on Jenkins and GitOps to an audience at KubeCon who I suspect are almost all interested in moving away from Jenkins, or have been told to be interested in switching from Jenkins to something else, and I tried to get the idea across that they probably don't really need to switch the workflow tool even if it's ancient, ... But maybe should consider subbing out some of the important fiddly bits underneath it (like, I assume the vast majority of Jenkins users are building images with Docker, and if they're running on Kubernetes, they're many of them wondering what they will do, or how long they really have before they have to start worrying about the deprecation of dockershim and how their lives are going to change when their clusters won't be running Docker under the hood anymore?) I was actually arguing for a tool like Porter.sh to come in and make the boundaries of "what's in a build" super neat and tidy, organized, but also limited so the next time they feel compelled to switch workflow tooling, it will be a non-issue and can be over and finished inside of a single day's work. The problem isn't that your workflow tool is too old, it's that you've jammed too much arbitrary complexity inside of it, probably because the right abstraction was not made available to you at the time. "Switching off of Jenkins" just means building a second system and it comes with all the baggage of "second system syndrome" to do so. Sure it's difficult switching from Jenkins, when you've built this gigantic pipeline with 18 branches and 12 of them run in parallel, half of them are configured with different options passed to the docker build tool, half of them must use buildx, and the third half of them are unmaintained so we don't go in there... so put some guard rails up around the hard parts! And get somebody in there to take care of those cobwebs.

ghotli 5y ago

I too love "Go by Example" and refer to it often. Makes me want it for all the things. Shot in the dark, anyone know of one for hot-off-the-presses modern Python (3.10) with typing akin to Golang? All the modern additions really need a comprehensive overview like Go by Example somehow manages to do in a very lightweight style

Kreotiko 5y ago

Word of advice, if you are thinking of running self hosted runners and use Actions for your organisation, do yourself a favour and check them out in a year or two and use something like Argo Workflows or Tekton instead. GHA isn’t a product thought for GH private organisations, you will find that every much needed feature for this use is very low in GH roadmap.

longcommonname 5y ago
Could you provide more details?
- Kreotiko 5y ago
  Sure, the main things for me are: It doesn’t matter how smart you are with reusable workflows you will never get to a truly DRY setup that scales for dozens of repositories. Another major pain is that we still haven’t private actions. It was due end of 2021 (maybe it is out now but I checked a couple of days ago). Setting up runners to look after a pool of repositories needs elevated permissions. GH offers a way to enforce a list of enabled actions but this does not work with private binary registries hosting pre built Docker actions. The only thing that could prevent you to pull software at runtime from the internet, which means, if you want to have a decent security posture all you are left with is referencing actions using the full git sha version. Many common use case require hacks, which is fun for a weekend project but isn’t great for a large scale operation. An example is simply running a workflow dynamically targeting the folders containing changes. At the moment you have to create a job, generate a build matrix on the fly and pass it in input as the matrix to the actual job.
  - sverhagen 5y ago
    Don't you have access to actions in private repositories, or is that not the same?
    - bewuethr 5y ago
      The only way I'm aware of to use a private action is to clone the repository it's in using a personal access token, and then use a local relative path to the action to run it.

filleokus 5y ago

GitHub Actions with act to test stuff locally is actually pretty usable. We use on-prem k8s-hosted runners to get access to servers/clusters with limited internet access, works great. But as always with “”gitops”” themed tools I think it’s pretty awkward to handle rollbacks. Either you take the stance that master is the source of truth and let the CI/CD tooling revert commits if deployment fails, or you store that state elsewhere and allow e.g manifests to diverge from git. I would be curious to hear how other people do it

bilalq 5y ago
Like all terms in the industry, "gitops" gets defined a little differently by everyone. In our case, we use trunk-based development where "main" is the branch that is approved for release, but it does not necessarily point to what is currently released. Instead, we use git tags for that. On merge to main, we kick off pipeline executions (we use AWS CodePipeline here, but Github actions with a concurrency group id set would achieve something similar). Non-prod stages just push lightweight tags in the format of `$STAGE-YYYY.MM.DD.hhmm` on successful deployment. Prod stages are similar, but we publish a Github release declaration using the Github API rather than just pushing a git tag. In the case of rollback, you either push new tags at older commits or roll forward with a revert commit (but still using tags as the tracking mechanism).
- ynouri 5y ago
  Thanks for sharing! We are a small company and right now we release (from our monorepo) on every PR merge to main. As the engineering team continues growing, we anticipate this approach will become unsustainable soon, and are looking for alternatives. I wonder if the tag based release is a widespread industry pattern (especially for growing monorepos)? Would be curious to learn more about it
  - DrewRWx 5y ago
    I'm on a team that implemented a hybrid tag-based release system on a monorepo and it is working well. External releases are built on release branches off of main with their own git tag rules, but I want to touch on internal releases off of main. Between feature branches and main we have GitHub status checks that gate bad PRs from getting in. Once the PRs are in main, we build a nightly full suite of components and put them through various levels of interop testing. Once our system receives the signal that all builds and interop testing had passed, it applies a Last Known Good (LKG) tag to the git commit the components were built from. After that, various systems including artifact links for QA and auto-merge jobs from main, are set to use the LKG tag to ensure they are getting good builds and code.

baby 5y ago

This is great! As feedback I would try to really enforce best practices. For example, every steps should have a descriptive name (not just “uses”)

devops000 5y ago

Cool. This should be added to the offical documentation.