3 ms·

Whenever possible I now just use GitHub actions as a thin wrapper around a Makefile and this has improved my experience with it a lot. The Makefile takes care o

by JanMa 2y ago

Whenever possible I now just use GitHub actions as a thin wrapper around a Makefile and this has improved my experience with it a lot. The Makefile takes care of installing all necessary dependencies and runs the relevant build/Test commands. This also enables me to test that stuff locally again without the long feedback loop mentioned in other comments in this thread.

oulipo 2y ago
mise (https://mise.jdx.dev/ https://mise.jdx.dev/) and dagger (https://github.com/dagger/dagger https://github.com/dagger/dagger) seem like nice candidates too! Mise can install all your deps, and run tasks
- lou1306 2y ago
  > * Don't bind yourself to some fancy new VC-financed thing that will solve CI once and for all but needs to get monetized eventually (see: earthly, dagger, etc.) Literally from comment at the root of this thread.
  - triyambakam 2y ago
    But are mise and dagger VC funded? I don't see any pricing pages there.
    - deleted 2y ago
      [deleted]
    - deng 2y ago
      Dagger is even YCombinator funded. https://www.boringbusinessnerd.com/startups/dagger https://www.boringbusinessnerd.com/startups/dagger Mise indeed isn't, but its scope is quite a bit smaller than Dagger.
      - internetter 2y ago
        > Mise indeed isn't, but its scope is quite a bit smaller than Dagger. A lot of us could learn... do one thing and do it well
        arcanemachiner 2y ago
        Ironic, because mise is a glued-together combination of asdf, direnv, and Makefiles.
  - verdverm 2y ago
    Docker has raised money, we all use it. Dagger is by the originators of Docker, I personally feel comfortable relying on them, they are making revenues too.
- jimmcslim 2y ago
  In addition to the other comments suggesting dagger is not the saviour due to being VC-funded, it seems like they have decided there's no money in CI, but AI... yes there's money there! And "something something agents". From dagger.io... "The open platform for agentic software. Build powerful, controllable agents on an open ecosystem. Deploy agentic applications with complete visibility and cross-language capabilities in a modular, extensible platform. Use Dagger to modernize your CI, customize AI workflows, build MCP servers, or create incredible agents."
  - __float 2y ago
    I can't really fault them too much for hopping on the latest bandwagon, if their software is general enough at running workflows for it to fit. They do seem to have a nice "quickstart for CI" they haven't abandoned, yet: https://docs.dagger.io/ci/quickstart https://docs.dagger.io/ci/quickstart (As much as I personally like working with CI and build systems, it's true there's not a ton of money in it!)
  - shykes 2y ago
    Hello! Dagger CEO here. Yes, we discovered that, in addition to running CI pipelines, Dagger can run AI agents. We learned this because our own users have told us. So now we are trying to capitalize on it, hence the ongoing changes to our website. We are trying to avoid the "something something agents" effect, but clearly, we still have work to do there :) It's hard to explain in marketing terms why a ephemeral execution engine, cross-language component system, deep observability and interactive CLI can be great at running both types of workloads... But we're going to keep trying! Internally we never thought of ourselves as a CI company, but as an operating system company operating in the CI market. Now we are expanding opportunistically to a new market: AI agents. We will continue to support both, because our platform can run both. If you are interested, I shared more details here: https://x.com/solomonstre/status/1895671390176747682 https://x.com/solomonstre/status/1895671390176747682
    - colemannerd 2y ago
      Please be careful. I'd love to adopt Dagger, but the UI in comparison to GHA, is just not a value add. I'd hate for y'all to go the AI route that Arc did... and lose all your users. There is A LOT to CICD, which can be profitable. I think there's still a lot more features needed before it's compelling and I would worry Agentic AI will lead you to a hyper-configurable, muddled message.
mwenge 2y ago
Do you have a public example of this? I'd love to see how to do this with Github Actions.
- cmsj 2y ago
  I don't have a makefile example, but I do functionally the same thing with shell scripts. I let GitHub actions do things like the initial environment configuration and the post-run formatting/annotation, but all of the actual work is done by my scripts: https://github.com/Hammerspoon/hammerspoon/blob/master/.github/workflows/ci_testbuild.yml#L30 https://github.com/Hammerspoon/hammerspoon/blob/master/.gith...
- JanMa 2y ago
  Sure, here's one example: https://github.com/JanMa/nomad-driver-nspawn/blob/master/.github/workflows/go.yml https://github.com/JanMa/nomad-driver-nspawn/blob/master/.gi...
- williamcotton 2y ago
  It doesn't (perhaps yet?) install the dependencies from the Makefile, but it runs a number of commands from the Makefile, eg, make test-leaks: https://github.com/williamcotton/webdsl/blob/main/.github/workflows/test.yml#L63 https://github.com/williamcotton/webdsl/blob/main/.github/wo...
akanapuli 2y ago
I dont quite understand the benefit. How does running commands from the Makefile differ from running commands directly on the runner ? What benefit does Makefile brings here ?
- ZeWaka 2y ago
  You can't run GitHub actions yml workflows locally (officially, there's tools like act).
- fiddlerwoaroof 2y ago
  If you have your CI runner use the same commands as local dev, CI basically becomes an integration test for the dev workflow. This also solves the “broken setup instructions” problem.
fireflash38 2y ago
I implemented a thing such that the makefiles locally use the same podman/docker images as the CI/CD uses. Every command looks something like: target: $(DOCKER_PREFIX) build When run in gitlab, the DOCKER_PREFIX is a no-op (it's literally empty due to the CI=true var), and the 'build' command (whatever it is) runs in the CI/CD docker image. When run locally, it effectively is a `docker run -v $(pwd):$(pwd) build`. It's really convenient for ensuring that if it builds locally, it can build in CI/CD.

by JanMa 2y ago

oulipo 2y ago

mise (https://mise.jdx.dev/ https://mise.jdx.dev/) and dagger (https://github.com/dagger/dagger https://github.com/dagger/dagger) seem like nice candidates too! Mise can install all your deps, and run tasks

lou1306 2y ago
> * Don't bind yourself to some fancy new VC-financed thing that will solve CI once and for all but needs to get monetized eventually (see: earthly, dagger, etc.) Literally from comment at the root of this thread.
- triyambakam 2y ago
  But are mise and dagger VC funded? I don't see any pricing pages there.
  - deleted 2y ago
    [deleted]
  - deng 2y ago
    Dagger is even YCombinator funded. https://www.boringbusinessnerd.com/startups/dagger https://www.boringbusinessnerd.com/startups/dagger Mise indeed isn't, but its scope is quite a bit smaller than Dagger.
    - internetter 2y ago
      > Mise indeed isn't, but its scope is quite a bit smaller than Dagger. A lot of us could learn... do one thing and do it well
      - arcanemachiner 2y ago
        Ironic, because mise is a glued-together combination of asdf, direnv, and Makefiles.
- verdverm 2y ago
  Docker has raised money, we all use it. Dagger is by the originators of Docker, I personally feel comfortable relying on them, they are making revenues too.
jimmcslim 2y ago
In addition to the other comments suggesting dagger is not the saviour due to being VC-funded, it seems like they have decided there's no money in CI, but AI... yes there's money there! And "something something agents". From dagger.io... "The open platform for agentic software. Build powerful, controllable agents on an open ecosystem. Deploy agentic applications with complete visibility and cross-language capabilities in a modular, extensible platform. Use Dagger to modernize your CI, customize AI workflows, build MCP servers, or create incredible agents."
- __float 2y ago
  I can't really fault them too much for hopping on the latest bandwagon, if their software is general enough at running workflows for it to fit. They do seem to have a nice "quickstart for CI" they haven't abandoned, yet: https://docs.dagger.io/ci/quickstart https://docs.dagger.io/ci/quickstart (As much as I personally like working with CI and build systems, it's true there's not a ton of money in it!)
- shykes 2y ago
  Hello! Dagger CEO here. Yes, we discovered that, in addition to running CI pipelines, Dagger can run AI agents. We learned this because our own users have told us. So now we are trying to capitalize on it, hence the ongoing changes to our website. We are trying to avoid the "something something agents" effect, but clearly, we still have work to do there :) It's hard to explain in marketing terms why a ephemeral execution engine, cross-language component system, deep observability and interactive CLI can be great at running both types of workloads... But we're going to keep trying! Internally we never thought of ourselves as a CI company, but as an operating system company operating in the CI market. Now we are expanding opportunistically to a new market: AI agents. We will continue to support both, because our platform can run both. If you are interested, I shared more details here: https://x.com/solomonstre/status/1895671390176747682 https://x.com/solomonstre/status/1895671390176747682
  - colemannerd 2y ago
    Please be careful. I'd love to adopt Dagger, but the UI in comparison to GHA, is just not a value add. I'd hate for y'all to go the AI route that Arc did... and lose all your users. There is A LOT to CICD, which can be profitable. I think there's still a lot more features needed before it's compelling and I would worry Agentic AI will lead you to a hyper-configurable, muddled message.

mwenge 2y ago

Do you have a public example of this? I'd love to see how to do this with Github Actions.

cmsj 2y ago
I don't have a makefile example, but I do functionally the same thing with shell scripts. I let GitHub actions do things like the initial environment configuration and the post-run formatting/annotation, but all of the actual work is done by my scripts: https://github.com/Hammerspoon/hammerspoon/blob/master/.github/workflows/ci_testbuild.yml#L30 https://github.com/Hammerspoon/hammerspoon/blob/master/.gith...
JanMa 2y ago
Sure, here's one example: https://github.com/JanMa/nomad-driver-nspawn/blob/master/.github/workflows/go.yml https://github.com/JanMa/nomad-driver-nspawn/blob/master/.gi...
williamcotton 2y ago
It doesn't (perhaps yet?) install the dependencies from the Makefile, but it runs a number of commands from the Makefile, eg, make test-leaks: https://github.com/williamcotton/webdsl/blob/main/.github/workflows/test.yml#L63 https://github.com/williamcotton/webdsl/blob/main/.github/wo...

akanapuli 2y ago

I dont quite understand the benefit. How does running commands from the Makefile differ from running commands directly on the runner ? What benefit does Makefile brings here ?

ZeWaka 2y ago
You can't run GitHub actions yml workflows locally (officially, there's tools like act).
fiddlerwoaroof 2y ago
If you have your CI runner use the same commands as local dev, CI basically becomes an integration test for the dev workflow. This also solves the “broken setup instructions” problem.

fireflash38 2y ago

I implemented a thing such that the makefiles locally use the same podman/docker images as the CI/CD uses. Every command looks something like: target: $(DOCKER_PREFIX) build When run in gitlab, the DOCKER_PREFIX is a no-op (it's literally empty due to the CI=true var), and the 'build' command (whatever it is) runs in the CI/CD docker image. When run locally, it effectively is a `docker run -v $(pwd):$(pwd) build`. It's really convenient for ensuring that if it builds locally, it can build in CI/CD.