14 ms·

Gitlab can load its ci file from a remote web server. What I have done is generate the yaml programmatically and this gets returned from the web server. Less th

Gitlab can load its ci file from a remote web server. What I have done is generate the yaml programmatically and this gets returned from the web server. Less than ideal but it at least allows dynamic creation

StavrosK 5y ago
You can just have a CI job generate a CI config file and have GitLab CI load that into a child pipeline. Look up dynamic pipelines.
- lyjackal 5y ago
  Ya, as I understand, that's all this python ci lib is doing. I find it a little annoying that your first have to start a worker to create the pipeline file, and then again to run the pipelines steps
  - remram 5y ago
    In this day and age it's not that weird. GitHub's own staff are spawning CI jobs just to add a label to a each new issue https://github.blog/2021-04-28-use-github-actions-manage-docs/ https://github.blog/2021-04-28-use-github-actions-manage-doc...
- nerdbaggy 5y ago
  I feel like there was some weird edge cases I was having with child pipelines. Maybe it was something with artifacts, don’t remember.
  - VOBOSHI 5y ago
    Artifacts, passing environment variables from the main pipeline to the child pipeline, expanding variable references in the child pipeline and getting unexpected results... We're migrating our shared pipeline away from child pipelines. Maybe one day we'll do a write-up. :)
- dnsmichi 5y ago
  I came across this great blog post: https://www.objectif-libre.com/en/blog/2021/02/23/a-new-era-for-gitlab-ci-dynamic-child-pipelines/ https://www.objectif-libre.com/en/blog/2021/02/23/a-new-era-... Documentation: https://docs.gitlab.com/ee/ci/parent_child_pipelines.html https://docs.gitlab.com/ee/ci/parent_child_pipelines.html GitLab 13.10 also added support for parallel matrix job execution in child pipelines, speeding up the execution once more. https://about.gitlab.com/releases/2021/03/22/gitlab-13-10-released/#use-parallel-matrix-with-trigger-jobs https://about.gitlab.com/releases/2021/03/22/gitlab-13-10-re... There's more to dynamic pipeline creation, collecting blog post ideas in https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/11122#note_552365143 https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/11122#... :)
- VOBOSHI 5y ago
  Our shared pipeline used to create child pipelines and we wouldn't recommend them. If all the shared pipeline jobs show up in your consumer project pipeline, you can easily extend them or overwrite them entirely.

StavrosK 5y ago

You can just have a CI job generate a CI config file and have GitLab CI load that into a child pipeline. Look up dynamic pipelines.

lyjackal 5y ago
Ya, as I understand, that's all this python ci lib is doing. I find it a little annoying that your first have to start a worker to create the pipeline file, and then again to run the pipelines steps
- remram 5y ago
  In this day and age it's not that weird. GitHub's own staff are spawning CI jobs just to add a label to a each new issue https://github.blog/2021-04-28-use-github-actions-manage-docs/ https://github.blog/2021-04-28-use-github-actions-manage-doc...
nerdbaggy 5y ago
I feel like there was some weird edge cases I was having with child pipelines. Maybe it was something with artifacts, don’t remember.
- VOBOSHI 5y ago
  Artifacts, passing environment variables from the main pipeline to the child pipeline, expanding variable references in the child pipeline and getting unexpected results... We're migrating our shared pipeline away from child pipelines. Maybe one day we'll do a write-up. :)
dnsmichi 5y ago
I came across this great blog post: https://www.objectif-libre.com/en/blog/2021/02/23/a-new-era-for-gitlab-ci-dynamic-child-pipelines/ https://www.objectif-libre.com/en/blog/2021/02/23/a-new-era-... Documentation: https://docs.gitlab.com/ee/ci/parent_child_pipelines.html https://docs.gitlab.com/ee/ci/parent_child_pipelines.html GitLab 13.10 also added support for parallel matrix job execution in child pipelines, speeding up the execution once more. https://about.gitlab.com/releases/2021/03/22/gitlab-13-10-released/#use-parallel-matrix-with-trigger-jobs https://about.gitlab.com/releases/2021/03/22/gitlab-13-10-re... There's more to dynamic pipeline creation, collecting blog post ideas in https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/11122#note_552365143 https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/11122#... :)
VOBOSHI 5y ago
Our shared pipeline used to create child pipelines and we wouldn't recommend them. If all the shared pipeline jobs show up in your consumer project pipeline, you can easily extend them or overwrite them entirely.