4 ms·

I spent a year building a distributed CI service, so I've thought about this a lot. To be honest I would really recommend giving either Gitlab CI or Github act

I spent a year building a distributed CI service, so I've thought about this a lot. To be honest I would really recommend giving either Gitlab CI or Github actions a shot. One of the main factors in my thing failing was the inability to compete with the vertical integration these tools both have, which is a massive lever for simplicity, power and reliability. The disadvantage you rightly highlight is some degree of lock in, should you ever need to move your code, but if you put the core logic of your workflows into scripts (which you likely should do regardless) then this really isn't a big issue. Genuinely, give Github actions a try - see if your opinion doesn't change once you see its advantages.

dec0dedab0de 5y ago
I have too much of an aversion to vendor lock-in and/or Microsoft. I don't think I could give it an honest appraisal. The only way I can see myself using it is if it becomes a mandate at work, and even then I'll make sure the project gets assigned to someone else.
mikepurvis 5y ago
My org has a mix of Jenkins and GitLab CI, and I think it actually works reasonably well, especially if you can have both systems targeting a common pool of k8s-managed compute. Basically there are large build/integration/test jobs that are centrally managed on Jenkins via Pipeline/Jenkinsfile, but then we offer a library of relatively simple common-case gitlab-ci configs that can be trivially pulled in, so most common cases are basically a one-liner (build dpkg, build setuptools, build sphinx, run nose, etc), but from the dev's point of view it's a self-serve thing rather than some mystery-meat process that can only be altered by filing DevOps tickets and hoping for the best. TBH I think my biggest complaint with GitLab CI is the insistence that every job is a brand new container. This means if you, say, have a big NPM project that installs thousands of dependencies on every run, you have to either a) hit your repo multiple times and not worry about it, b) figure out a local-enough caching proxy for your repo that it doesn't matter, c) combine your build/test/lint stages into a single one, d) build from a nightly container that has a bunch of it pre-installed, e) abuse the artifacts system to pass the workspace between jobs, f) spend multiple hours trying to make the supplied caching mechanism work, only to find that no matter what you do, NPM ignores the cache and re-fetches every time, and even if it worked, you'd have to pick between various non-ideal cache-keying strategies: https://docs.gitlab.com/ee/ci/caching/ https://docs.gitlab.com/ee/ci/caching/
- oweiler 5y ago
  The caching is Gitlab CI biggest weakness IMHO, super slow, and as you said cache strategies are to few to be useful.
  - Koffiepoeder 5y ago
    Tip: Using the fast zip feature flag when configuring runners/pipelines can help quite a bit https://docs.gitlab.com/runner/configuration/feature-flags.html https://docs.gitlab.com/runner/configuration/feature-flags.h...
    - oweiler 5y ago
      I've tried this out but the difference was almost immeasurable. Probably depends on the type of project.

dec0dedab0de 5y ago

I have too much of an aversion to vendor lock-in and/or Microsoft. I don't think I could give it an honest appraisal. The only way I can see myself using it is if it becomes a mandate at work, and even then I'll make sure the project gets assigned to someone else.

mikepurvis 5y ago

My org has a mix of Jenkins and GitLab CI, and I think it actually works reasonably well, especially if you can have both systems targeting a common pool of k8s-managed compute. Basically there are large build/integration/test jobs that are centrally managed on Jenkins via Pipeline/Jenkinsfile, but then we offer a library of relatively simple common-case gitlab-ci configs that can be trivially pulled in, so most common cases are basically a one-liner (build dpkg, build setuptools, build sphinx, run nose, etc), but from the dev's point of view it's a self-serve thing rather than some mystery-meat process that can only be altered by filing DevOps tickets and hoping for the best. TBH I think my biggest complaint with GitLab CI is the insistence that every job is a brand new container. This means if you, say, have a big NPM project that installs thousands of dependencies on every run, you have to either a) hit your repo multiple times and not worry about it, b) figure out a local-enough caching proxy for your repo that it doesn't matter, c) combine your build/test/lint stages into a single one, d) build from a nightly container that has a bunch of it pre-installed, e) abuse the artifacts system to pass the workspace between jobs, f) spend multiple hours trying to make the supplied caching mechanism work, only to find that no matter what you do, NPM ignores the cache and re-fetches every time, and even if it worked, you'd have to pick between various non-ideal cache-keying strategies: https://docs.gitlab.com/ee/ci/caching/ https://docs.gitlab.com/ee/ci/caching/

oweiler 5y ago
The caching is Gitlab CI biggest weakness IMHO, super slow, and as you said cache strategies are to few to be useful.
- Koffiepoeder 5y ago
  Tip: Using the fast zip feature flag when configuring runners/pipelines can help quite a bit https://docs.gitlab.com/runner/configuration/feature-flags.html https://docs.gitlab.com/runner/configuration/feature-flags.h...
  - oweiler 5y ago
    I've tried this out but the difference was almost immeasurable. Probably depends on the type of project.