4 ms·

I'm not a gitlab employee or anything, but I'm curious: How is gitlab CI not flexible enough for you? I mean, it basically just runs arbitrary docker container

I'm not a gitlab employee or anything, but I'm curious: How is gitlab CI not flexible enough for you? I mean, it basically just runs arbitrary docker containers. From my mostly amateur (when it comes to this, i'm a professional developer but I don't manage our CI system directly) point of view it seems like the gitlab CI is more flexible than jenkins not less.

sytse 10y ago
As you indicated we hope that GitLab CI covers all use cases but we're always interested in learning things we can improve.
- elygre 10y ago
  I added a comment to the parent, regarding test reporting. It's a showstopper for us.
mikepurvis 10y ago
The strong coupling between repos and CI jobs is a huge obstacle— for a multi-repo codebase, you need to have triggers that are outside of "a new commit has been pushed". The reporting is a big problem too— Jenkins has plugins which format and report on JUnit-style reports, including displaying images or other assets generated by tests. Configuration. The config needing to be committed to the repo is an issue. For many use cases including ours, it's way better to have the CI configuration stored centrally somewhere, and deployed using Jenkins Job Builder (or the various other similar tools which exist for Jenkins). Executor control. Jenkins lets you do things like put N slots on a buildslave and then have a job occupy M of them, as well as controlling what jobs can run where, and what the priority of different jobs is. This is critical for ensuring overall sanity and efficiency in the pipeline. Jenkins isn't perfect, but it happens to be a better fit for our needs, and these four factors are all a big part of that.
- sytse 10y ago
  For multiple projects you can use build triggers https://docs.gitlab.com/ce/ci/triggers/README.html https://docs.gitlab.com/ce/ci/triggers/README.html We're considering parsing JUnit style tests https://gitlab.com/gitlab-org/gitlab-ce/issues/22098 https://gitlab.com/gitlab-org/gitlab-ce/issues/22098 We think storing the configuration in the repository makes it easier to collaborate on it (new branch is tested in the new way, master in the old way). Can you share your use case for this? You can specify how many jobs you want GitLab Runner to run. It can even spin up more VMs automatically if you want that. You can bind jobs to runners with labels and specific runners https://docs.gitlab.com/ee/ci/runners/README.html https://docs.gitlab.com/ee/ci/runners/README.html
  - mikepurvis 10y ago
    Our use-case is perhaps a bit unique, but ROS has a federated development model, meaning literally hundreds of repos. See the following file which maps repos to package names and versions: https://github.com/ros/rosdistro/blob/master/indigo/distribution.yaml https://github.com/ros/rosdistro/blob/master/indigo/distribu... These get generated into thousands of jobs: http://build.ros.org/view/All/ http://build.ros.org/view/All/ It would be totally unrealistic to rev job configurations by making commits into the individual repos (and in the public open source case, actually impossible— the maintainers of that community build infrastructure don't have write access to many of the repos— it's equivalent to a Debian distro using DSC files to grab tarballs).
    - sytse 10y ago
      Thanks for the use case and the links. I must admit I never saw a 14234 line yaml configuration file before. The jobs page is not loading for me right now. I think that in general developers and the build engineers should both have commit access. But I see your dilemma and think we need to have a conversation to see what the possibilities are.
- Snappy 10y ago
  Great feedback, thanks! This isn't exactly what you asked for, but we're seriously considering letting you "include" CI configuration from a central location. https://gitlab.com/gitlab-org/gitlab-ce/issues/20868 https://gitlab.com/gitlab-org/gitlab-ce/issues/20868
elygre 10y ago
As one example, GitLab CI does not have a good story with handling unit test failures from e.g. gradle. You can choose to keep the build output from a failed build, which you can download as a zip file, unpack locally, and look at the reports. This user experience makes test reports mostly useless. At the same time, they have test coverage badges, where test output is parsed to provide coverage. To me, that's really weird: You don't get proper access to test failures, but lots of work into test coverage. (I have no knowledge of this, but I suspect that being a Ruby-shop, they have little experience with working with java projects, and therefore don't really understand how the toolchain works. This is all guesswork, though). In the end, we cannot give up Jenkins. We need to know the result of our tests, and see the test output!
- sytse 10y ago
  I totally agree that we can do better showing test output and we're working on that in https://gitlab.com/gitlab-org/gitlab-ce/issues/22098 https://gitlab.com/gitlab-org/gitlab-ce/issues/22098
merb 10y ago
most people probably miss JUnit-style reports.
- sytse 10y ago
  Yep, see https://gitlab.com/gitlab-org/gitlab-ce/issues/22098 https://gitlab.com/gitlab-org/gitlab-ce/issues/22098 for the feature proposal.
mitchty 10y ago
Just running docker containers is basically a show stopper for certain use cases. Not every use case can even fit into a docker container. E.g. testing kernel modules.
- sytse 10y ago
  Totally agree that not everything works on Docker. GitLab Runner works great on metal (Linux, OSX, Windows, RPi, etc.), no need to use Docker.

sytse 10y ago

As you indicated we hope that GitLab CI covers all use cases but we're always interested in learning things we can improve.

elygre 10y ago
I added a comment to the parent, regarding test reporting. It's a showstopper for us.

mikepurvis 10y ago

The strong coupling between repos and CI jobs is a huge obstacle— for a multi-repo codebase, you need to have triggers that are outside of "a new commit has been pushed". The reporting is a big problem too— Jenkins has plugins which format and report on JUnit-style reports, including displaying images or other assets generated by tests. Configuration. The config needing to be committed to the repo is an issue. For many use cases including ours, it's way better to have the CI configuration stored centrally somewhere, and deployed using Jenkins Job Builder (or the various other similar tools which exist for Jenkins). Executor control. Jenkins lets you do things like put N slots on a buildslave and then have a job occupy M of them, as well as controlling what jobs can run where, and what the priority of different jobs is. This is critical for ensuring overall sanity and efficiency in the pipeline. Jenkins isn't perfect, but it happens to be a better fit for our needs, and these four factors are all a big part of that.

sytse 10y ago
For multiple projects you can use build triggers https://docs.gitlab.com/ce/ci/triggers/README.html https://docs.gitlab.com/ce/ci/triggers/README.html We're considering parsing JUnit style tests https://gitlab.com/gitlab-org/gitlab-ce/issues/22098 https://gitlab.com/gitlab-org/gitlab-ce/issues/22098 We think storing the configuration in the repository makes it easier to collaborate on it (new branch is tested in the new way, master in the old way). Can you share your use case for this? You can specify how many jobs you want GitLab Runner to run. It can even spin up more VMs automatically if you want that. You can bind jobs to runners with labels and specific runners https://docs.gitlab.com/ee/ci/runners/README.html https://docs.gitlab.com/ee/ci/runners/README.html
- mikepurvis 10y ago
  Our use-case is perhaps a bit unique, but ROS has a federated development model, meaning literally hundreds of repos. See the following file which maps repos to package names and versions: https://github.com/ros/rosdistro/blob/master/indigo/distribution.yaml https://github.com/ros/rosdistro/blob/master/indigo/distribu... These get generated into thousands of jobs: http://build.ros.org/view/All/ http://build.ros.org/view/All/ It would be totally unrealistic to rev job configurations by making commits into the individual repos (and in the public open source case, actually impossible— the maintainers of that community build infrastructure don't have write access to many of the repos— it's equivalent to a Debian distro using DSC files to grab tarballs).
  - sytse 10y ago
    Thanks for the use case and the links. I must admit I never saw a 14234 line yaml configuration file before. The jobs page is not loading for me right now. I think that in general developers and the build engineers should both have commit access. But I see your dilemma and think we need to have a conversation to see what the possibilities are.
Snappy 10y ago
Great feedback, thanks! This isn't exactly what you asked for, but we're seriously considering letting you "include" CI configuration from a central location. https://gitlab.com/gitlab-org/gitlab-ce/issues/20868 https://gitlab.com/gitlab-org/gitlab-ce/issues/20868

elygre 10y ago

As one example, GitLab CI does not have a good story with handling unit test failures from e.g. gradle. You can choose to keep the build output from a failed build, which you can download as a zip file, unpack locally, and look at the reports. This user experience makes test reports mostly useless. At the same time, they have test coverage badges, where test output is parsed to provide coverage. To me, that's really weird: You don't get proper access to test failures, but lots of work into test coverage. (I have no knowledge of this, but I suspect that being a Ruby-shop, they have little experience with working with java projects, and therefore don't really understand how the toolchain works. This is all guesswork, though). In the end, we cannot give up Jenkins. We need to know the result of our tests, and see the test output!

sytse 10y ago
I totally agree that we can do better showing test output and we're working on that in https://gitlab.com/gitlab-org/gitlab-ce/issues/22098 https://gitlab.com/gitlab-org/gitlab-ce/issues/22098

merb 10y ago

most people probably miss JUnit-style reports.

sytse 10y ago
Yep, see https://gitlab.com/gitlab-org/gitlab-ce/issues/22098 https://gitlab.com/gitlab-org/gitlab-ce/issues/22098 for the feature proposal.

mitchty 10y ago

Just running docker containers is basically a show stopper for certain use cases. Not every use case can even fit into a docker container. E.g. testing kernel modules.

sytse 10y ago
Totally agree that not everything works on Docker. GitLab Runner works great on metal (Linux, OSX, Windows, RPi, etc.), no need to use Docker.