5 ms·

Dear Gitlab... please focus on finishing out your parent/child pipeline concepts. I'm very disappointed to see basic CI capabilities missing in 2023. None of th

by ellisd 3y ago

Dear Gitlab... please focus on finishing out your parent/child pipeline concepts. I'm very disappointed to see basic CI capabilities missing in 2023. None of the shiny AI things matter if CI isn't solid. Today parent pipelines cannot consume any test report from the child pipeline using the Gitlab CI DSL: https://gitlab.com/groups/gitlab-org/-/epics/8205 https://gitlab.com/groups/gitlab-org/-/epics/8205 We would effectively need to roll up the child pipeline's results manually in the parent pipeline by calling APIs ourselves.

klysm 3y ago
AI fomo takes priority sorry
andy_ppp 3y ago
Gitlab CI is so weird man, depending on where you are workflow/rules and so on seems to have very different effects. The obsession it is has making jobs === VMs is also mind bending, why so many virtual machines for one pipeline! You can't consolidate child pipeline reporting up to a higher level - all the data is there in the UI but hidden. There were loads of other things. Github Actions which is largely free (if you use your own runners) is much better.
- CameronNemo 3y ago
  What do you mean by jobs == VMs? I thought gitlab ci was mostly containers.
  - hhh 3y ago
    You choose what you want it to be. https://docs.gitlab.com/runner/executors/ https://docs.gitlab.com/runner/executors/
    - CameronNemo 3y ago
      Sure, but the hosted runners use the docker executor IIRC and container-based executors are quite common.
- andrewstuart2 3y ago
  Not sure what you're getting at, as I've always found GitLab CI substantially more obvious than actions. CI definitely shouldn't be a snowflake, and it almost certainly shouldn't be something you can't run locally with very similar steps. GitHub makes it nearly impossible to replicate a CI build locally unless you're super familiar with the actions being used and how they were implemented, because actions are highly abstracted from what is actually happening. GitLab CI is just a series of shell commands with a container image for context (bring your own tools). It's usually pretty easy to figure out a GitLab build unless you get fancy with includes or child jobs which imo, for the aforementioned reasons, are substantial antipatterns.
  - mathstuf 3y ago
    The other things I really like about GitLab's CI is that it's easy to use different images/machines for different tasks. For example, the build of a CUDA-using job can be done on any hardware, but the testing needs CUDA access. Instead of tying up our CUDA machines on boring compiles, the jobs are split on build/test lines so that the CUDA hardware has a better shot of actually being utilized well. Same thing with giving only testing access to things like X11 setups or whatnot. One can also build once and test multiple ways with this method (e.g., build a ABI3-using Python wheel once and test it with all newer versions of Python with distinct jobs to spread the work and to allow the pipeline to show better things at a glance. I have no idea how to do such "tee" bits in GitHub Actions (though I haven't looked in a while, the artifacts usage seem…convoluted compared to GitLab's "here's a list of globs"). For that matter, "join"s would be nice too (e.g., to collect the 20 or so wheels built across various jobs into one `twine` upload).
kyrofa 3y ago
Yeah I finally just stopped watching that issue. I would get a few emails a week that is just some gitlab sales person saying "another customer interested in this" over and over. It's become pretty obvious that it has lost its internal champion, whoever that was. Parent/child was very close to a very cool feature.
Sayrus 3y ago
I've been following that Epic and the issue it is based on since 2021 (which is how I found your comment). It's been a roller-coaster which grew from small issues to a giant epic. A few times a week, I get an email about it hoping it's finally a patch for the artifact type I'm using. In the end, I manually roll up all artifacts to the parent pipeline and have a MRs ready in case the feature is released. There are quite a few issues I follow, sometimes they are very small, sometimes they are huge epics. While I love GitLab and GitLab CI, I find myself hitting some of these frustrating issues nearly every time I do something uncommon. They usually end up being delivered and well done, which is to the credit of the awesome folks at GitLab, but they rarely take less than a few years.
- brodock 3y ago
  Have you considered sending a MR for the "very small" things that bugs you? GitLab team member here... I have a personal list of things I wish will be prioritized, but there is only much you can do with the scope the application has and the amount of coworkers. So when things really piss me off, I reserve some time and send a MR to fix that. That's what I've done before joining the company, so it works ;)
andrewstuart2 3y ago
Trying to treat CICD as a function interface, that is calling other jobs with inputs that return outputs, is so sketchy to me. I've been tasked with unraveling these types of constructions before and after that experience, I'd personally prefer anything of the sort be deprecated entirely unless it's given dramatically better dev tooling. At that point you're probably not creating builds, you're creating apps, and you should just use an actual programming language that supports tests and debugging, and pull that into your pipelines as a proper tool with proper controls and its own build.

by ellisd 3y ago

klysm 3y ago

AI fomo takes priority sorry

andy_ppp 3y ago

Gitlab CI is so weird man, depending on where you are workflow/rules and so on seems to have very different effects. The obsession it is has making jobs === VMs is also mind bending, why so many virtual machines for one pipeline! You can't consolidate child pipeline reporting up to a higher level - all the data is there in the UI but hidden. There were loads of other things. Github Actions which is largely free (if you use your own runners) is much better.

CameronNemo 3y ago
What do you mean by jobs == VMs? I thought gitlab ci was mostly containers.
- hhh 3y ago
  You choose what you want it to be. https://docs.gitlab.com/runner/executors/ https://docs.gitlab.com/runner/executors/
  - CameronNemo 3y ago
    Sure, but the hosted runners use the docker executor IIRC and container-based executors are quite common.
andrewstuart2 3y ago
Not sure what you're getting at, as I've always found GitLab CI substantially more obvious than actions. CI definitely shouldn't be a snowflake, and it almost certainly shouldn't be something you can't run locally with very similar steps. GitHub makes it nearly impossible to replicate a CI build locally unless you're super familiar with the actions being used and how they were implemented, because actions are highly abstracted from what is actually happening. GitLab CI is just a series of shell commands with a container image for context (bring your own tools). It's usually pretty easy to figure out a GitLab build unless you get fancy with includes or child jobs which imo, for the aforementioned reasons, are substantial antipatterns.
- mathstuf 3y ago
  The other things I really like about GitLab's CI is that it's easy to use different images/machines for different tasks. For example, the build of a CUDA-using job can be done on any hardware, but the testing needs CUDA access. Instead of tying up our CUDA machines on boring compiles, the jobs are split on build/test lines so that the CUDA hardware has a better shot of actually being utilized well. Same thing with giving only testing access to things like X11 setups or whatnot. One can also build once and test multiple ways with this method (e.g., build a ABI3-using Python wheel once and test it with all newer versions of Python with distinct jobs to spread the work and to allow the pipeline to show better things at a glance. I have no idea how to do such "tee" bits in GitHub Actions (though I haven't looked in a while, the artifacts usage seem…convoluted compared to GitLab's "here's a list of globs"). For that matter, "join"s would be nice too (e.g., to collect the 20 or so wheels built across various jobs into one `twine` upload).

kyrofa 3y ago

Yeah I finally just stopped watching that issue. I would get a few emails a week that is just some gitlab sales person saying "another customer interested in this" over and over. It's become pretty obvious that it has lost its internal champion, whoever that was. Parent/child was very close to a very cool feature.

Sayrus 3y ago

I've been following that Epic and the issue it is based on since 2021 (which is how I found your comment). It's been a roller-coaster which grew from small issues to a giant epic. A few times a week, I get an email about it hoping it's finally a patch for the artifact type I'm using. In the end, I manually roll up all artifacts to the parent pipeline and have a MRs ready in case the feature is released. There are quite a few issues I follow, sometimes they are very small, sometimes they are huge epics. While I love GitLab and GitLab CI, I find myself hitting some of these frustrating issues nearly every time I do something uncommon. They usually end up being delivered and well done, which is to the credit of the awesome folks at GitLab, but they rarely take less than a few years.

brodock 3y ago
Have you considered sending a MR for the "very small" things that bugs you? GitLab team member here... I have a personal list of things I wish will be prioritized, but there is only much you can do with the scope the application has and the amount of coworkers. So when things really piss me off, I reserve some time and send a MR to fix that. That's what I've done before joining the company, so it works ;)

andrewstuart2 3y ago

Trying to treat CICD as a function interface, that is calling other jobs with inputs that return outputs, is so sketchy to me. I've been tasked with unraveling these types of constructions before and after that experience, I'd personally prefer anything of the sort be deprecated entirely unless it's given dramatically better dev tooling. At that point you're probably not creating builds, you're creating apps, and you should just use an actual programming language that supports tests and debugging, and pull that into your pipelines as a proper tool with proper controls and its own build.