5 ms·

It seems like a core argument is the pre-commit tests that runs as commit hooks on the developers computer. I have worked in a place where they did that, and I

It seems like a core argument is the pre-commit tests that runs as commit hooks on the developers computer. I have worked in a place where they did that, and I think the cons heavily outweighed the pros. I can not push incomplete work to a remote, I can not separate feature development and chores (eg. linting) because I _need_ to fix stuff in order to commit an push, etc. > Continuous Integration is a model thats tightly tied to how we run version control. I would say that a pre-commit testing world is much tighter. CI, as many know it, as a passive observer. When stuff fails, it will let you know. You can still merge. One thing that would be nice, however, would be the ability to run the entire pipeline locally. For GitHub actions, it indeed seems like there are viable options to do that.

rtpg 4y ago
I do feel like if we’re going the precommit route we need to go much deeper into things like Bazel. Loads of companies are using CI to get a huge test suite to run with a lot of parralelsim for a reason! Though for me it’s also like… so many people do unspeakable horrors to their machine setups that I like there being a consistent runner in CI. But “CI is to run tests” in the world of CD is a bit of a simplification anyways.
- masklinn 4y ago
  > Loads of companies are using CI to get a huge test suite to run with a lot of parralelsim for a reason! Tho technically you can probably have a local runner which either distributes the test suite across a bunch of available matchines (à la pytest-xdist), or one which goes and creates jobs on the CI platform without needing to go through the pretest ceremony (e.g. creating commits, branches, CRs, ...)
  - rtpg 4y ago
    Right, bazel lets you do that (it has no notion of version control state), and ships it off remotel if you so choose to have the system work like that. I stand by this idea that if someone made Bazel, but without all the obtuseness, with strong integration to a hosted CI service, they would quickly overtake a lot of services.
runeks 4y ago
> One thing that would be nice, however, would be the ability to run the entire pipeline locally. For GitHub actions, it indeed seems like there are viable options to do that. I prefer not having to replicate locally how GitHub runs GitHub actions, but rather just make my GitHub actions run something that I know I can run locally. So all the complicated stuff is contained in the script that I can run locally, and GitHub actions is just a dumb “runner” of this script. For my local script I prefer using Nix, since it’s the best way I know to guarantee a consistent environment between different machines.
- nathants 4y ago
  this is the way.
- diarrhea 4y ago
  That where I use Makefiles. For GitLab CI, for example, you can have the likes of `make $CI_JOB_NAME` so your CI config can be very short and dry, just a bunch of jobs named after Makefile targets.
__alexs 4y ago
git has cli flags that will happily let you bypass pre commit/push checks so it's not even a reliable way to know that tests actually got run. Central CI gives you a traceable test history of every change and keeps us all honest.
- madsbuch 4y ago
  This might break the teams assumption that these things pass. After all, centralized CI has, in this narrative, been exchanged for the assumption that everything pushed to git passes all tests.
wereHamster 4y ago
I `export HUSKY=0` in all my shells because pre-commit hooks are stupid.
- verve_rat 4y ago
  Thanks for that. I agree that pre-commit hooks enforce one way of working that just wastes a bunch of my time.
Epa095 4y ago
We use nektos act to run pipelines locally. Works OK, but everytime something fails locally you are left wondering if its act or your pipeline, and some features (like reusable workflows) are not implemented yet. Maybe the right thing is to just write the workflows in something else, and have the github workflow file be a single call to your script. But -It would be nice to be able to use github actions others has made (or libraries/abstractions in general) - I don't see how to easily get parallell execution. - I love github environments, how do I pass down all my environments and their variables?
- southerntofu 4y ago
  Does it work 100% offline if the docker images have been cached?
ownagefool 4y ago
I think pre-commits are useful, but the prediction is basically "trust the client". What if the client is compromised? Are we throwing away reviews? Should the reviewer re-run all the tests?
lvncelot 4y ago
> One thing that would be nice, however, would be the ability to run the entire pipeline locally. This cost me many hours of waiting for the Gitlab CI runner when debugging non-trivial pipelines, when the issue was something that did not have to do with the script steps inside of the jobs but rather how the Gitlab runner handled things. I've found gitlab-ci-local [1] which actually does run the Gitlab pipeline locally, although I had to write some boilerplaye scripts to set up all the necessary 'CI_FOO_SOMETHING' environment variables before running the tool. (Which sometimes came back to bite me because the issue was actually in the content of some of those environment variables). It's still a very good tool. [1] https://github.com/firecow/gitlab-ci-local https://github.com/firecow/gitlab-ci-local
- declnz 4y ago
  Thanks for the link Side note: makes me sad when even open-source tools for Gitlab are hosted... on Github...
citrin_ru 4y ago
It sounds like a problem with the policy, not the tool. Pre-commit hooks is a useful tool but there should be a way to skip them if you really need it. Per-commit hooks save my time by not allowing to push code which guaranteed to break later in CI.
mpweiher 4y ago
> pre-commit tests that runs as commit hook Also hate those. What I do instead is have the tests run as part of the build. If the tests fail, the build fails. Definitely a good motivator for having fast tests, which is essential. The ones for my projects tend to run in about a second or two. Also gives you way greater confidence in your tests. Try it! https://blog.metaobject.com/2020/05/mpwtest-reducing-test-friction-by-going.html https://blog.metaobject.com/2020/05/mpwtest-reducing-test-fr... https://blog.metaobject.com/2020/06/mpwtest-only-tests-frameworks.html https://blog.metaobject.com/2020/06/mpwtest-only-tests-frame...
reillyse 4y ago
"One thing that would be nice, however, would be the ability to run the entire pipeline locally." I'm working on something to do just this. Although I've redefined what "local" means in this context. I'm still using remote servers but everything is happening pre-commit from the terminal. If you are interested check out this demo and let me know if you have any feedback https://brisktest.com/demos https://brisktest.com/demos

rtpg 4y ago

I do feel like if we’re going the precommit route we need to go much deeper into things like Bazel. Loads of companies are using CI to get a huge test suite to run with a lot of parralelsim for a reason! Though for me it’s also like… so many people do unspeakable horrors to their machine setups that I like there being a consistent runner in CI. But “CI is to run tests” in the world of CD is a bit of a simplification anyways.

masklinn 4y ago
> Loads of companies are using CI to get a huge test suite to run with a lot of parralelsim for a reason! Tho technically you can probably have a local runner which either distributes the test suite across a bunch of available matchines (à la pytest-xdist), or one which goes and creates jobs on the CI platform without needing to go through the pretest ceremony (e.g. creating commits, branches, CRs, ...)
- rtpg 4y ago
  Right, bazel lets you do that (it has no notion of version control state), and ships it off remotel if you so choose to have the system work like that. I stand by this idea that if someone made Bazel, but without all the obtuseness, with strong integration to a hosted CI service, they would quickly overtake a lot of services.

runeks 4y ago

> One thing that would be nice, however, would be the ability to run the entire pipeline locally. For GitHub actions, it indeed seems like there are viable options to do that. I prefer not having to replicate locally how GitHub runs GitHub actions, but rather just make my GitHub actions run something that I know I can run locally. So all the complicated stuff is contained in the script that I can run locally, and GitHub actions is just a dumb “runner” of this script. For my local script I prefer using Nix, since it’s the best way I know to guarantee a consistent environment between different machines.

nathants 4y ago
this is the way.
diarrhea 4y ago
That where I use Makefiles. For GitLab CI, for example, you can have the likes of `make $CI_JOB_NAME` so your CI config can be very short and dry, just a bunch of jobs named after Makefile targets.

__alexs 4y ago

git has cli flags that will happily let you bypass pre commit/push checks so it's not even a reliable way to know that tests actually got run. Central CI gives you a traceable test history of every change and keeps us all honest.

madsbuch 4y ago
This might break the teams assumption that these things pass. After all, centralized CI has, in this narrative, been exchanged for the assumption that everything pushed to git passes all tests.

wereHamster 4y ago

I `export HUSKY=0` in all my shells because pre-commit hooks are stupid.

verve_rat 4y ago
Thanks for that. I agree that pre-commit hooks enforce one way of working that just wastes a bunch of my time.

Epa095 4y ago

We use nektos act to run pipelines locally. Works OK, but everytime something fails locally you are left wondering if its act or your pipeline, and some features (like reusable workflows) are not implemented yet. Maybe the right thing is to just write the workflows in something else, and have the github workflow file be a single call to your script. But -It would be nice to be able to use github actions others has made (or libraries/abstractions in general) - I don't see how to easily get parallell execution. - I love github environments, how do I pass down all my environments and their variables?

southerntofu 4y ago
Does it work 100% offline if the docker images have been cached?

ownagefool 4y ago

I think pre-commits are useful, but the prediction is basically "trust the client". What if the client is compromised? Are we throwing away reviews? Should the reviewer re-run all the tests?

lvncelot 4y ago

> One thing that would be nice, however, would be the ability to run the entire pipeline locally. This cost me many hours of waiting for the Gitlab CI runner when debugging non-trivial pipelines, when the issue was something that did not have to do with the script steps inside of the jobs but rather how the Gitlab runner handled things. I've found gitlab-ci-local [1] which actually does run the Gitlab pipeline locally, although I had to write some boilerplaye scripts to set up all the necessary 'CI_FOO_SOMETHING' environment variables before running the tool. (Which sometimes came back to bite me because the issue was actually in the content of some of those environment variables). It's still a very good tool. [1] https://github.com/firecow/gitlab-ci-local https://github.com/firecow/gitlab-ci-local

declnz 4y ago
Thanks for the link Side note: makes me sad when even open-source tools for Gitlab are hosted... on Github...

citrin_ru 4y ago

It sounds like a problem with the policy, not the tool. Pre-commit hooks is a useful tool but there should be a way to skip them if you really need it. Per-commit hooks save my time by not allowing to push code which guaranteed to break later in CI.

mpweiher 4y ago

> pre-commit tests that runs as commit hook Also hate those. What I do instead is have the tests run as part of the build. If the tests fail, the build fails. Definitely a good motivator for having fast tests, which is essential. The ones for my projects tend to run in about a second or two. Also gives you way greater confidence in your tests. Try it! https://blog.metaobject.com/2020/05/mpwtest-reducing-test-friction-by-going.html https://blog.metaobject.com/2020/05/mpwtest-reducing-test-fr... https://blog.metaobject.com/2020/06/mpwtest-only-tests-frameworks.html https://blog.metaobject.com/2020/06/mpwtest-only-tests-frame...

reillyse 4y ago

"One thing that would be nice, however, would be the ability to run the entire pipeline locally." I'm working on something to do just this. Although I've redefined what "local" means in this context. I'm still using remote servers but everything is happening pre-commit from the terminal. If you are interested check out this demo and let me know if you have any feedback https://brisktest.com/demos https://brisktest.com/demos