4 ms·

Exporting environment variables via artifacts sounds like a strange decision, but I guess its one way to do it. After years of waiting for jobs to communicate o

Exporting environment variables via artifacts sounds like a strange decision, but I guess its one way to do it. After years of waiting for jobs to communicate other then relying on file artifacts this is somewhat disappointing. I would like to see parent/child pipelines recive some love as currently it does work but quirks are all around. For example, its not easy (or sometimes even possible) to pass pipeline variables from parent to child, pipline UI behaves differently when being part of relation (and many times unusuable or shows a wrong thing), not being able to repeat manual jobs with the same var initially passed, not even being able to run it again with any other var unless you delete all previous executions, strange limitations on masked secret vars, $ in your password will get evaluated as variable etc ... Seems like an afterthought, rather then a carefully designed feature set. Too organic for my taste (I guess Conway's law is full blown there). While Gitlab CI is getting better (or more capable, rather then better) then before in every release, it does seem bloated, overly complex, full of surprises, not reproducible locally, very slow (caching is ridiculous feature that makes job often run longer then without it) and with strange design that doesn't let me view my build log full screen or collect entire pipeline output easily. Generally, Windows is also lagging behind (I can't use pwsh as runner shell today after 4 years in existence?). The worst recent offender: you display 'fail' on every job in color: "WARNING: Failed to load system CertPool: crypto/x509: system root pool is not available on Windows". This trips everybody that job has failed when it didn't. I think Gitlab guys need to start taking CI/CD/runner more seriously or at least bring some fresh mind to it. After all, this is one of the major reasons for many people to use it.

oweiler 6y ago
The caching is so broken it's barely usable. You can't easily exclude files. The cache can't be updated incrementally. It's often slower than downloading all dependencies on every build.
- latch 6y ago
  Instead of caching node_modules, cache node_modules.tar.szt Then wrap your job in a untar/tar: tar --use-compress-program zstd -xf node_modules.tar.zst || true YOUR NORMAL JOB tar --use-compress-program zstd -cf node_modules.tar.zst node_modules (and you can delete things you don't need if you want)
  - Already__Taken 6y ago
    That's weird won't these all be docker layers somewhere in the guts? So it's already compressed.
- orf 6y ago
  The caching is not broken and it's completely usable. It sounds like you've got a weird specific use case that doesn't benefit from a general solution.
  - morelisp 6y ago
    The caching has worked great for me for Java / JVM languages, and decently for Go (but a proxy is usually better). It is unusable for NPM or composer, or the version of yarn we use (maybe newer ones too but I know they were trying to improve their internal cache a while back). Anything with a ton of small files or entire git histories. It’s awful, and it’s not weird or specific.
tyeager 6y ago
Announced today as part of our 13.0 Release is the ability to pass variables/data between jobs without using artifacts - https://about.gitlab.com/releases/2020/05/22/gitlab-13-0-released/#inherit-environment-variables-from-other-jobs https://about.gitlab.com/releases/2020/05/22/gitlab-13-0-rel.... For a list of planned follow-up issues as we iterating on the parent-child pipeline MVC, please check out our epic for this feature: https://gitlab.com/groups/gitlab-org/-/epics/2750 https://gitlab.com/groups/gitlab-org/-/epics/2750. We welcome your comments and up-votes on the issues that matter most to you. EDIT: Sorry; misread your first sentence; seems you saw the announcement that there is an alternative to passing variable via artifacts. Please consider opening an issue (https://gitlab.com/gitlab-org/gitlab/-/issues/new https://gitlab.com/gitlab-org/gitlab/-/issues/new) to let us know how you would like jobs to communication.
Grollicus 6y ago
They still haven't fixed process killing (that is, not sending SIGKILL immediately but sending SIGTERM so that child processes get cleaned up and the runner doesn't get stuck waiting for child processes to exit that don't know they should exit). For being THE feature that made gitlab big the whole CI area seems to be getting away from them

oweiler 6y ago

The caching is so broken it's barely usable. You can't easily exclude files. The cache can't be updated incrementally. It's often slower than downloading all dependencies on every build.

latch 6y ago
Instead of caching node_modules, cache node_modules.tar.szt Then wrap your job in a untar/tar: tar --use-compress-program zstd -xf node_modules.tar.zst || true YOUR NORMAL JOB tar --use-compress-program zstd -cf node_modules.tar.zst node_modules (and you can delete things you don't need if you want)
- Already__Taken 6y ago
  That's weird won't these all be docker layers somewhere in the guts? So it's already compressed.
orf 6y ago
The caching is not broken and it's completely usable. It sounds like you've got a weird specific use case that doesn't benefit from a general solution.
- morelisp 6y ago
  The caching has worked great for me for Java / JVM languages, and decently for Go (but a proxy is usually better). It is unusable for NPM or composer, or the version of yarn we use (maybe newer ones too but I know they were trying to improve their internal cache a while back). Anything with a ton of small files or entire git histories. It’s awful, and it’s not weird or specific.

tyeager 6y ago

Announced today as part of our 13.0 Release is the ability to pass variables/data between jobs without using artifacts - https://about.gitlab.com/releases/2020/05/22/gitlab-13-0-released/#inherit-environment-variables-from-other-jobs https://about.gitlab.com/releases/2020/05/22/gitlab-13-0-rel.... For a list of planned follow-up issues as we iterating on the parent-child pipeline MVC, please check out our epic for this feature: https://gitlab.com/groups/gitlab-org/-/epics/2750 https://gitlab.com/groups/gitlab-org/-/epics/2750. We welcome your comments and up-votes on the issues that matter most to you. EDIT: Sorry; misread your first sentence; seems you saw the announcement that there is an alternative to passing variable via artifacts. Please consider opening an issue (https://gitlab.com/gitlab-org/gitlab/-/issues/new https://gitlab.com/gitlab-org/gitlab/-/issues/new) to let us know how you would like jobs to communication.

Grollicus 6y ago

They still haven't fixed process killing (that is, not sending SIGKILL immediately but sending SIGTERM so that child processes get cleaned up and the runner doesn't get stuck waiting for child processes to exit that don't know they should exit). For being THE feature that made gitlab big the whole CI area seems to be getting away from them