8 ms·

Show HN: Test your Gitlab CI Pipelines changes locally using Docker

83 points by damnhotuser 6y ago

auraham 6y ago
Interesting repo! It would be great to have more documentation since I am a beginner in Docker.
- brudgers 6y ago
  Not personal, maybe your unfamiliarity with Docker means you're not the target audience. I mean getting people up to speed with Docker is a bit of an ocean to boil, and this project appears to be small of resources...
ypeter 6y ago
Cool, I'm gonna test this out. If it works well it's really useful.
Aeolun 6y ago
This looks interesting. Will test it out. Hope it works with includes.
- damnhotuser 6y ago
  Yeah it does work with includes. One of the pain point right now is that it does not work with cache:untracked or artifacts:untracked, but I'm trying to figure it out :)
  - Aeolun 6y ago
    Haha, I have nothing that relies on that, so should be fine for me. I guess you do something like secretly make separate artifact folders per job and keep copying (or just linking) stuff around?
Nullabillity 6y ago
Why not just use GitLab's runner[0] directly for this? gitlab-runner exec docker <job> [0]: https://docs.gitlab.com/runner/install/ https://docs.gitlab.com/runner/install/
- damnhotuser 6y ago
  Yeah I thought about that, but gitlab-runner doesn't allow to "share" cache or artifacts between jobs so it's not a good fit to test multiple-jobs workflows.
  - vergessenmir 6y ago
    Haven't test the tool yet. Neither do I know how close the behaviour of the pipeline locally will differ on the server. Saying that, pipeline test tools are very much needed. Making commits and refreshing a UI is not practical. I can test a distributed pipeline on spark why can't I do that with modern CI/CD?
  - jayd16 6y ago
    Sure it does. Jobs have a shared cache (although this can be slow) and artifacts are shared to downstream jobs. Maybe this is a tiered feature? https://docs.gitlab.com/ee/ci/caching/ https://docs.gitlab.com/ee/ci/caching/
    - sjburt 6y ago
      When using the "exec" command it doesn't do that. It just executes one job, not the whole pipeline.
- carlosf 6y ago
  exec is extremely limited and it's not being actively maintained.
obayesshelton 6y ago
Great idea. One question - what pain point is this solving? As a GitLab, GitLab Pipeline and Docker user I don't find running pipelines that slow to run or initiate
- conradludgate 6y ago
  Personally with Github Actions, I do find myself pushing 2, 3 or 4 times just getting the pipeline working because I always mess something up, either I've messed up a command or the environment isn't what I expected etc. There's act[0] for github which I should use more often to solve this, but I usually forget [0]: https://github.com/nektos/act https://github.com/nektos/act
- wdb 6y ago
  You continuously need to push your changes to change `.gitlab-ci.yml` and find out if your change is working? E.g. your docker run command. I haven't tried out the tool yet but I hope it will save time and avoid me pushing 100s of commits to sort out my pipeline changes.
- carlosf 6y ago
  The problem is not necessarily slow init, but some pipelines are just long and complex. If you have a problem in your deployment script, for example, you might need to wait a long time to test and fix it.
- manojlds 6y ago
  Why do you say great idea when you didn't even get the pain point it is solving?
- powerapple 6y ago
  From what I understand, you use it as a test for your pipeline scripts. With code we run unittest locally to make sure our commits are okay. When develop ci scripts, many times you only find out things don't work until you push the ci scripts to GitLab, it leaves you with many try and fix commits in history.
donlebon 6y ago
Great! I am currently working in an Azure Devops project. We would definitely need such a tool!
frwickst 6y ago
Thanks for a neat project! I tried it on one of our more complex pipelines and created a few issues for you ;)
time0ut 6y ago
This looks awesome. I can’t wait to try it. GitLab CI is great, but tweaking it until its right is the worst part of my workflow. Thank you!
Tyrubias 6y ago
This looks great! I hate it when I have to commit and push a new change for a simple bug. That being said, how does this differ from something like act [1] (besides the fact this is for Gitlab)? [1] https://github.com/nektos/act https://github.com/nektos/act
- mdaniel 6y ago
  > (besides the fact this is for Gitlab)? Heh, yeah, besides being a different programming language, how is node different from Java? :-D I, too, would value "the one CI yaml to rule them all" but given how much faster GitLab is moving than GitHub, there's almost no prayer of them trying to have some kind of common behavior. I actually had the best experience using circleci's local runner (back when we were using it for CI) as far as "run binary, perform build" goes. I have yet to see gitlab-runner successfully execute a local build :-(
rhn_mk1 6y ago
Nice! I made a similar tool a while back: https://gitlab.com/rhn/cirunn https://gitlab.com/rhn/cirunn It's a pair of really simple Python scripts: cirunn.py for executing locally, an ciorder.py to execute stuff via ssh on another host (useful for projects that are too big for the laptop). The main advantage is that podman does not need root access at all, making it both easier to use and more secure. Also, no installation. The difficulty I found here is deciding what the user actually means to run: is it more like `make test`, where the current state of the working directory is tested, or more like the CI pipeline, where only committed changes are tested? For the sake of practicality, the local script tests uncommitted changes, and the remote one checks the last commit.
carlosf 6y ago
As a GitLab / Docker user, testing pipelines is definitely a major pain point in my devops process. Cool stuff!
nickjj 6y ago
IMO you can do this without needing extra tooling if you keep your CI scripts as shell scripts and your application is running in Docker. If you keep your CI scripts as shell scripts and put them into a runnable file included in your project you can run your CI scripts locally so you can test the work flow on your machine. It's also handy in case CI is down you can still test and deploy your code. Lastly it lets you easily jump between CI providers since a majority of your logic is in a generic shell script. For example in my CI specific files I typically only call `./run ci:install-deps && ./run ci:test`. Then there's a tiny bit of boiler plate in each CI provider's yml file to handle specific things for that provider. If your app is running in Docker most of the heavy duty dependencies are all contained there. The dependencies I install directly in CI end up being things like installing shellcheck and helper scripts for CI (like tiny scripts that let you wait until a process is ready). Having WSL 2 is nice here because these are very small tools that I want locally installed anyways but even if you use macOS you could install these dependencies using brew instead. A working example of this is here: https://github.com/nickjj/docker-flask-example https://github.com/nickjj/docker-flask-example Check out the run script in the root of the project for "ci:" functions and the GH Actions yml file. This same strategy applies to GitLab or any other CI provider too.
- deleted 6y ago
  [deleted]
- xrendan 6y ago
  We do this but with makefiles to contain whatever CI logic which also makes it easy to switch.
- madjam002 6y ago
  I also like this approach, although I am using Nix instead of Docker for a slightly more lightweight way of managing dependencies of different stages. If you also reduce the assumptions made by your CI scripts, as in the scripts themselves will authenticate to and set up any needed connectivity / port forwarding, or fetch secrets instead of being passed secrets from the CI system, then you end up with some really nice portable scripts that can be run from anywhere. The only things that my CI scripts depend on is an authentication token to Hashicorp Vault, an internet connection and Nix to be installed on the machine.
julienlafond 6y ago
Excellent idea!
0x008 6y ago
any chance of running this without installing yarn?
jnwatson 6y ago
Having to push a commit to test the Github equivalent (Actions) was a huge pain. I think I ended up with 80+ commits. I wish the existing solutions on the Github side worked. Mildly ironic that you're hosting this on Github.
- mdaniel 6y ago
  > I wish the existing solutions on the Github side worked. I did experience a bad outcome using act[0] for a super weird matrix-y workflow, but I'd say for the most part it behaved sanely; did you report your complaints to their issue tracker? 0 = https://github.com/nektos/act https://github.com/nektos/act
  - jnwatson 6y ago
    I briefly tried it. It couldn’t even parse the yaml.
- benibela 6y ago
  > I think I ended up with 80+ commits. Happens all the time to me. (although I have not used Actions) . Especially when I also had to set up a cross compiler on the CI. And each change needs 15 minutes to run the VM again CI is an enormous productivity killer
- c7DJTLrn 6y ago
  Yep, CI like this is a catastrophe. Sourcehut's ad-hoc builds are really nice because you can test without pushing any changes. But sadly, Sourcehut CI is lacking in other areas.
mshekow 6y ago
How does this differ from Earthly? https://github.com/earthly/earthly https://github.com/earthly/earthly Earthly lets you abstract anything you do inside a container into an Earthfile, that runs locally on a dev's machine, but also in any CI, making CI scripts more portable. However, testing other CI-specific things before committing, such as GitLab rules dictating when jobs actually run, remain unsolved. But they also remain unsolved with this glci solution, right?

83 points by damnhotuser 6y ago

auraham 6y ago

Interesting repo! It would be great to have more documentation since I am a beginner in Docker.

brudgers 6y ago
Not personal, maybe your unfamiliarity with Docker means you're not the target audience. I mean getting people up to speed with Docker is a bit of an ocean to boil, and this project appears to be small of resources...

ypeter 6y ago

Cool, I'm gonna test this out. If it works well it's really useful.

Aeolun 6y ago

This looks interesting. Will test it out. Hope it works with includes.

damnhotuser 6y ago
Yeah it does work with includes. One of the pain point right now is that it does not work with cache:untracked or artifacts:untracked, but I'm trying to figure it out :)
- Aeolun 6y ago
  Haha, I have nothing that relies on that, so should be fine for me. I guess you do something like secretly make separate artifact folders per job and keep copying (or just linking) stuff around?

Nullabillity 6y ago

Why not just use GitLab's runner[0] directly for this? gitlab-runner exec docker <job> [0]: https://docs.gitlab.com/runner/install/ https://docs.gitlab.com/runner/install/

damnhotuser 6y ago
Yeah I thought about that, but gitlab-runner doesn't allow to "share" cache or artifacts between jobs so it's not a good fit to test multiple-jobs workflows.
- vergessenmir 6y ago
  Haven't test the tool yet. Neither do I know how close the behaviour of the pipeline locally will differ on the server. Saying that, pipeline test tools are very much needed. Making commits and refreshing a UI is not practical. I can test a distributed pipeline on spark why can't I do that with modern CI/CD?
- jayd16 6y ago
  Sure it does. Jobs have a shared cache (although this can be slow) and artifacts are shared to downstream jobs. Maybe this is a tiered feature? https://docs.gitlab.com/ee/ci/caching/ https://docs.gitlab.com/ee/ci/caching/
  - sjburt 6y ago
    When using the "exec" command it doesn't do that. It just executes one job, not the whole pipeline.
carlosf 6y ago
exec is extremely limited and it's not being actively maintained.

obayesshelton 6y ago

Great idea. One question - what pain point is this solving? As a GitLab, GitLab Pipeline and Docker user I don't find running pipelines that slow to run or initiate

conradludgate 6y ago
Personally with Github Actions, I do find myself pushing 2, 3 or 4 times just getting the pipeline working because I always mess something up, either I've messed up a command or the environment isn't what I expected etc. There's act[0] for github which I should use more often to solve this, but I usually forget [0]: https://github.com/nektos/act https://github.com/nektos/act
wdb 6y ago
You continuously need to push your changes to change `.gitlab-ci.yml` and find out if your change is working? E.g. your docker run command. I haven't tried out the tool yet but I hope it will save time and avoid me pushing 100s of commits to sort out my pipeline changes.
carlosf 6y ago
The problem is not necessarily slow init, but some pipelines are just long and complex. If you have a problem in your deployment script, for example, you might need to wait a long time to test and fix it.
manojlds 6y ago
Why do you say great idea when you didn't even get the pain point it is solving?
powerapple 6y ago
From what I understand, you use it as a test for your pipeline scripts. With code we run unittest locally to make sure our commits are okay. When develop ci scripts, many times you only find out things don't work until you push the ci scripts to GitLab, it leaves you with many try and fix commits in history.

donlebon 6y ago

Great! I am currently working in an Azure Devops project. We would definitely need such a tool!

frwickst 6y ago

Thanks for a neat project! I tried it on one of our more complex pipelines and created a few issues for you ;)

time0ut 6y ago

This looks awesome. I can’t wait to try it. GitLab CI is great, but tweaking it until its right is the worst part of my workflow. Thank you!

Tyrubias 6y ago

This looks great! I hate it when I have to commit and push a new change for a simple bug. That being said, how does this differ from something like act [1] (besides the fact this is for Gitlab)? [1] https://github.com/nektos/act https://github.com/nektos/act

mdaniel 6y ago
> (besides the fact this is for Gitlab)? Heh, yeah, besides being a different programming language, how is node different from Java? :-D I, too, would value "the one CI yaml to rule them all" but given how much faster GitLab is moving than GitHub, there's almost no prayer of them trying to have some kind of common behavior. I actually had the best experience using circleci's local runner (back when we were using it for CI) as far as "run binary, perform build" goes. I have yet to see gitlab-runner successfully execute a local build :-(

rhn_mk1 6y ago

Nice! I made a similar tool a while back: https://gitlab.com/rhn/cirunn https://gitlab.com/rhn/cirunn It's a pair of really simple Python scripts: cirunn.py for executing locally, an ciorder.py to execute stuff via ssh on another host (useful for projects that are too big for the laptop). The main advantage is that podman does not need root access at all, making it both easier to use and more secure. Also, no installation. The difficulty I found here is deciding what the user actually means to run: is it more like `make test`, where the current state of the working directory is tested, or more like the CI pipeline, where only committed changes are tested? For the sake of practicality, the local script tests uncommitted changes, and the remote one checks the last commit.

carlosf 6y ago

As a GitLab / Docker user, testing pipelines is definitely a major pain point in my devops process. Cool stuff!

nickjj 6y ago

IMO you can do this without needing extra tooling if you keep your CI scripts as shell scripts and your application is running in Docker. If you keep your CI scripts as shell scripts and put them into a runnable file included in your project you can run your CI scripts locally so you can test the work flow on your machine. It's also handy in case CI is down you can still test and deploy your code. Lastly it lets you easily jump between CI providers since a majority of your logic is in a generic shell script. For example in my CI specific files I typically only call `./run ci:install-deps && ./run ci:test`. Then there's a tiny bit of boiler plate in each CI provider's yml file to handle specific things for that provider. If your app is running in Docker most of the heavy duty dependencies are all contained there. The dependencies I install directly in CI end up being things like installing shellcheck and helper scripts for CI (like tiny scripts that let you wait until a process is ready). Having WSL 2 is nice here because these are very small tools that I want locally installed anyways but even if you use macOS you could install these dependencies using brew instead. A working example of this is here: https://github.com/nickjj/docker-flask-example https://github.com/nickjj/docker-flask-example Check out the run script in the root of the project for "ci:" functions and the GH Actions yml file. This same strategy applies to GitLab or any other CI provider too.

deleted 6y ago
[deleted]
xrendan 6y ago
We do this but with makefiles to contain whatever CI logic which also makes it easy to switch.
madjam002 6y ago
I also like this approach, although I am using Nix instead of Docker for a slightly more lightweight way of managing dependencies of different stages. If you also reduce the assumptions made by your CI scripts, as in the scripts themselves will authenticate to and set up any needed connectivity / port forwarding, or fetch secrets instead of being passed secrets from the CI system, then you end up with some really nice portable scripts that can be run from anywhere. The only things that my CI scripts depend on is an authentication token to Hashicorp Vault, an internet connection and Nix to be installed on the machine.

julienlafond 6y ago

Excellent idea!

0x008 6y ago

any chance of running this without installing yarn?

jnwatson 6y ago

Having to push a commit to test the Github equivalent (Actions) was a huge pain. I think I ended up with 80+ commits. I wish the existing solutions on the Github side worked. Mildly ironic that you're hosting this on Github.

mdaniel 6y ago
> I wish the existing solutions on the Github side worked. I did experience a bad outcome using act[0] for a super weird matrix-y workflow, but I'd say for the most part it behaved sanely; did you report your complaints to their issue tracker? 0 = https://github.com/nektos/act https://github.com/nektos/act
- jnwatson 6y ago
  I briefly tried it. It couldn’t even parse the yaml.
benibela 6y ago
> I think I ended up with 80+ commits. Happens all the time to me. (although I have not used Actions) . Especially when I also had to set up a cross compiler on the CI. And each change needs 15 minutes to run the VM again CI is an enormous productivity killer
c7DJTLrn 6y ago
Yep, CI like this is a catastrophe. Sourcehut's ad-hoc builds are really nice because you can test without pushing any changes. But sadly, Sourcehut CI is lacking in other areas.

mshekow 6y ago

How does this differ from Earthly? https://github.com/earthly/earthly https://github.com/earthly/earthly Earthly lets you abstract anything you do inside a container into an Earthfile, that runs locally on a dev's machine, but also in any CI, making CI scripts more portable. However, testing other CI-specific things before committing, such as GitLab rules dictating when jobs actually run, remain unsolved. But they also remain unsolved with this glci solution, right?