5 ms·
IMO you can do this without needing extra tooling if you keep your CI scripts as shell scripts and your application is running in Docker. If you keep your CI s
by nickjj 6y ago
IMO you can do this without needing extra tooling if you keep your CI scripts as shell scripts and your application is running in Docker.
If you keep your CI scripts as shell scripts and put them into a runnable file included in your project you can run your CI scripts locally so you can test the work flow on your machine. It's also handy in case CI is down you can still test and deploy your code. Lastly it lets you easily jump between CI providers since a majority of your logic is in a generic shell script.
For example in my CI specific files I typically only call `./run ci:install-deps && ./run ci:test`. Then there's a tiny bit of boiler plate in each CI provider's yml file to handle specific things for that provider.
If your app is running in Docker most of the heavy duty dependencies are all contained there. The dependencies I install directly in CI end up being things like installing shellcheck and helper scripts for CI (like tiny scripts that let you wait until a process is ready). Having WSL 2 is nice here because these are very small tools that I want locally installed anyways but even if you use macOS you could install these dependencies using brew instead.
A working example of this is here: https://github.com/nickjj/docker-flask-example https://github.com/nickjj/docker-flask-example
Check out the run script in the root of the project for "ci:" functions and the GH Actions yml file. This same strategy applies to GitLab or any other CI provider too.
- deleted 6y ago[deleted]
- xrendan 6y agoWe do this but with makefiles to contain whatever CI logic which also makes it easy to switch.
- madjam002 6y agoI also like this approach, although I am using Nix instead of Docker for a slightly more lightweight way of managing dependencies of different stages. If you also reduce the assumptions made by your CI scripts, as in the scripts themselves will authenticate to and set up any needed connectivity / port forwarding, or fetch secrets instead of being passed secrets from the CI system, then you end up with some really nice portable scripts that can be run from anywhere. The only things that my CI scripts depend on is an authentication token to Hashicorp Vault, an internet connection and Nix to be installed on the machine.
- mikewhy 6y agoSame, my CI has looked like this for years now: ./script/prepare-env ./script/test ./script/build ./script/deploy It's worked in virtually every CI system, and locally. Bonus point, it really points out the amount of ceremony some CI platforms require to map simple commands into their pipelines. Looking at you, CircleCI.
- mnahkies 6y agoI found an approach like this was especially nice on teamcity in conjunction with its template feature. Individual apps were still able to have whatever specific thing they needed, but you could globally add a step, eg a security scanning tool or slack notification by modifying the template only rather than 100s of build configuration files as the GitHub / circleci approach seems to encourage. Adding a new pipeline basically just meant naming it and selecting the VCS URL
- Already__Taken 6y agourgh no please don't wrap simple commands like docker build in shell called from makefile it's so f'ing complex to follow when it falls apart. If you're going to do script-in-script like this at least go to a proper cross-platform language like python, powershell or js anything but make+bash it's just the worst of all worlds.