4 ms·

Rather than tying CI & deployments to Github Actions, it is usually better to pull as much of it as possible out to shell scripts and call them in containers in

Rather than tying CI & deployments to Github Actions, it is usually better to pull as much of it as possible out to shell scripts and call them in containers in GH actions.. There are optimizations you’ll want (caching downloaded dependencies etc); if you wait to make those after your build is CI-agnostic you’ll be less tempted by vendor specific shortcuts. Usually means more code - but, easier to test locally. Also, swapping providers later will be difficult but not “spin up a team and spend 6 months” level.

Cloudef 1y ago
Nix is in fact perfect for this.
- lenova 1y ago
  As a non-Nix user, could you give me an example?
  - Cloudef 1y ago
    While flakes have their own "checks" thing, what I do is just have a app definition in flake with the required dependencies. https://github.com/Cloudef/zig-aio/blob/master/flake.nix#L25-L56 https://github.com/Cloudef/zig-aio/blob/master/flake.nix#L25... https://github.com/Cloudef/zig-budoux/blob/master/flake.nix#L30-L42 https://github.com/Cloudef/zig-budoux/blob/master/flake.nix#... The actual GA workflow file is pretty simple: https://github.com/Cloudef/zig-aio/blob/master/.github/workflows/pr.yml#L19-L30 https://github.com/Cloudef/zig-aio/blob/master/.github/workf...
jiggawatts 1y ago
This is always the top comment in these kinds of threads, and I see this as an indication that the current state of CI/CD is pathetically propriety. It’s like the dark times before free and open source compilers. When are we going to push back and say enough is enough!? CI/CD desperately needs something akin to Kubernetes to claw back our control and ability to work locally. Personally, I’m fed up with pipeline development inner loops that involve a Git commit, push, and waiting around for five minutes with no debugger, no variable inspector, and dumping things to console logs like I’m writing C in the 1980s. You and I shouldn’t be reinventing these wheels while standing inside the tyre shop.
- rsanheim 1y ago
  We've had open source CI. We still have it. I remember ye olde days of Hudson, before it was renamed Jenkins. Lotsa orgs still use Jenkins all over the place, it just doesn't get much press. It predates GHA, Circle, and many of the popular cloud offerings. Turns out CI/CD is not an easy problem. I built a short-lived CI product before containers were really much of a thing ...you can guess how well that went. Also, I'll take _any_ CI solution, closed or open, that tries to be the _opposite_ of the complexity borg that is k8s.
- nitwit005 1y ago
  Having a container makes debugging possible, but it's still generally going to be an unfriendly experience, compared to a script you can just run and debug immediately. It's inevitable that things will be more difficult to debug once you're using a third party asynchronous tool as part of the flow.
jbmsf 1y ago
That's mostly true. - You may need something to connect the dots between code changes and containers. It's not always possible to build everything on every change, especially in a multi/mono-repo setup. - You probably need some way to connect container outcomes back to branch protection rules. Again, if you are building everything, every time, it's pretty simple, but less so otherwise. - You likely want to have some control over the compute capacity on which the actions run, both for speed and cost control. And since caching matters, some compute solutions are better than others. I don't think GitHub Actions solves any of these problems well, but neither do containers on their own.

Cloudef 1y ago

Nix is in fact perfect for this.

lenova 1y ago
As a non-Nix user, could you give me an example?
- Cloudef 1y ago
  While flakes have their own "checks" thing, what I do is just have a app definition in flake with the required dependencies. https://github.com/Cloudef/zig-aio/blob/master/flake.nix#L25-L56 https://github.com/Cloudef/zig-aio/blob/master/flake.nix#L25... https://github.com/Cloudef/zig-budoux/blob/master/flake.nix#L30-L42 https://github.com/Cloudef/zig-budoux/blob/master/flake.nix#... The actual GA workflow file is pretty simple: https://github.com/Cloudef/zig-aio/blob/master/.github/workflows/pr.yml#L19-L30 https://github.com/Cloudef/zig-aio/blob/master/.github/workf...

jiggawatts 1y ago

This is always the top comment in these kinds of threads, and I see this as an indication that the current state of CI/CD is pathetically propriety. It’s like the dark times before free and open source compilers. When are we going to push back and say enough is enough!? CI/CD desperately needs something akin to Kubernetes to claw back our control and ability to work locally. Personally, I’m fed up with pipeline development inner loops that involve a Git commit, push, and waiting around for five minutes with no debugger, no variable inspector, and dumping things to console logs like I’m writing C in the 1980s. You and I shouldn’t be reinventing these wheels while standing inside the tyre shop.

rsanheim 1y ago
We've had open source CI. We still have it. I remember ye olde days of Hudson, before it was renamed Jenkins. Lotsa orgs still use Jenkins all over the place, it just doesn't get much press. It predates GHA, Circle, and many of the popular cloud offerings. Turns out CI/CD is not an easy problem. I built a short-lived CI product before containers were really much of a thing ...you can guess how well that went. Also, I'll take _any_ CI solution, closed or open, that tries to be the _opposite_ of the complexity borg that is k8s.
nitwit005 1y ago
Having a container makes debugging possible, but it's still generally going to be an unfriendly experience, compared to a script you can just run and debug immediately. It's inevitable that things will be more difficult to debug once you're using a third party asynchronous tool as part of the flow.

jbmsf 1y ago

That's mostly true. - You may need something to connect the dots between code changes and containers. It's not always possible to build everything on every change, especially in a multi/mono-repo setup. - You probably need some way to connect container outcomes back to branch protection rules. Again, if you are building everything, every time, it's pretty simple, but less so otherwise. - You likely want to have some control over the compute capacity on which the actions run, both for speed and cost control. And since caching matters, some compute solutions are better than others. I don't think GitHub Actions solves any of these problems well, but neither do containers on their own.