4 ms·

I took part in evaluating Concourse CI for the needs of my company (30+ devs). While it has amazing CI pipeline capabilities, we ultimately didn't select Concou

by witten 9y ago

I took part in evaluating Concourse CI for the needs of my company (30+ devs). While it has amazing CI pipeline capabilities, we ultimately didn't select Concourse because it felt much more like a CI toolbox, requiring some development to put those tools to use. And what we really wanted was more of a turnkey CI product. Perhaps ironically, we ended up doing some development around the edges of the CI product we ultimately selected (GitLab).

sytse 9y ago
Cool to hear you selected GitLab. We recently added multi project pipeline visualization. That was partially inspired by concourse CI. Is there anything else you would like to see?
- witten 9y ago
  Interesting you should mention that. Right around the time you implemented multi-project pipeline visualization, we built a bot that listens for GitLab build completion events, hits the (undocumented /unsupported) internal GitLab global code search ElasticSearch index, finds downstream dependencies of the completed build (as declared in setup.py/package.json), and triggers their builds as well.. Taking full advantage of your nifty visualization. So I think the feature we'd want to make that easier is an actual global code search API!
  - sytse 9y ago
    Cool! I think we're building this into GitLab itself. I'll ask the CI CD team to comment.
  - grzesiekb 9y ago
    Thanks for this proposal! It might be a cool feature indeed. I created an issue about it in our issue tracker, see https://gitlab.com/gitlab-org/gitlab-ce/issues/35186 https://gitlab.com/gitlab-org/gitlab-ce/issues/35186. We do not have immediate plans to ship that yet, but I pinged our product team member there. However we do have plans to ship multi-project pipeline with an "inversion of control". You will be able to specify pipeline relation with an upstream project, and when someone pushes a commit to it, a downstream pipeline is going to be trigger automatically. See an issue about cross-project dependencies - https://gitlab.com/gitlab-org/gitlab-ee/issues/1681 https://gitlab.com/gitlab-org/gitlab-ee/issues/1681!
    - witten 9y ago
      Awesome, thanks for filing!
- gaff33 9y ago
  Please can you allow for any acyclic graph rather than the current stage system? This would allow us to have our long running integration tests run in parallel with shorter tasks! I wrote an issue for it here: https://gitlab.com/gitlab-org/gitlab-ce/issues/29347 https://gitlab.com/gitlab-org/gitlab-ce/issues/29347
mariusmg 9y ago
>requiring some development to put those tools to use The best CI is the one where you write code to build/deploy etc. Wrangling with some moronic UI exposed by the CI product to abstract code is simply wrong. Don't do CI/CD like this.
- gabrielr 9y ago
  Right. One of my favorite parts of Concourse is that all the config is declarative, in YAML that teams can (and do) check into a public Git repo (with the secrets kept elsewhere). That lets teams share Concourse task and resource definitions. Many teams (including mine) publish our pipeline definitions too [0]. This enables "tooling" type teams build components that get a lot of re-use by other teams. disclaimer: I work for Pivotal. [0] e.g. code.cloudfoundry.org/cf-networking-ci
- witten 9y ago
  Some development is fine (and expected). And indeed the goal of "build automation" was what led us to evaluate both Concourse and GitLab. We were previously on Bamboo, which really did require wrangling with a moronic UI. But I think there's a difference between a mostly complete CI product with API extension points, and a "batteries not included" CI toolbox.
jacques_chester 9y ago
> While it has amazing CI pipeline capabilities, we ultimately didn't select Concourse because it felt much more like a CI toolbox, requiring some development to put those tools to use. This is actually a brilliant insight. I'd never thought of it that way. And I'm a one-eyed raving Concourse fan.

by witten 9y ago

sytse 9y ago

Cool to hear you selected GitLab. We recently added multi project pipeline visualization. That was partially inspired by concourse CI. Is there anything else you would like to see?

witten 9y ago
Interesting you should mention that. Right around the time you implemented multi-project pipeline visualization, we built a bot that listens for GitLab build completion events, hits the (undocumented /unsupported) internal GitLab global code search ElasticSearch index, finds downstream dependencies of the completed build (as declared in setup.py/package.json), and triggers their builds as well.. Taking full advantage of your nifty visualization. So I think the feature we'd want to make that easier is an actual global code search API!
- sytse 9y ago
  Cool! I think we're building this into GitLab itself. I'll ask the CI CD team to comment.
- grzesiekb 9y ago
  Thanks for this proposal! It might be a cool feature indeed. I created an issue about it in our issue tracker, see https://gitlab.com/gitlab-org/gitlab-ce/issues/35186 https://gitlab.com/gitlab-org/gitlab-ce/issues/35186. We do not have immediate plans to ship that yet, but I pinged our product team member there. However we do have plans to ship multi-project pipeline with an "inversion of control". You will be able to specify pipeline relation with an upstream project, and when someone pushes a commit to it, a downstream pipeline is going to be trigger automatically. See an issue about cross-project dependencies - https://gitlab.com/gitlab-org/gitlab-ee/issues/1681 https://gitlab.com/gitlab-org/gitlab-ee/issues/1681!
  - witten 9y ago
    Awesome, thanks for filing!
gaff33 9y ago
Please can you allow for any acyclic graph rather than the current stage system? This would allow us to have our long running integration tests run in parallel with shorter tasks! I wrote an issue for it here: https://gitlab.com/gitlab-org/gitlab-ce/issues/29347 https://gitlab.com/gitlab-org/gitlab-ce/issues/29347

mariusmg 9y ago

>requiring some development to put those tools to use The best CI is the one where you write code to build/deploy etc. Wrangling with some moronic UI exposed by the CI product to abstract code is simply wrong. Don't do CI/CD like this.

gabrielr 9y ago
Right. One of my favorite parts of Concourse is that all the config is declarative, in YAML that teams can (and do) check into a public Git repo (with the secrets kept elsewhere). That lets teams share Concourse task and resource definitions. Many teams (including mine) publish our pipeline definitions too [0]. This enables "tooling" type teams build components that get a lot of re-use by other teams. disclaimer: I work for Pivotal. [0] e.g. code.cloudfoundry.org/cf-networking-ci
witten 9y ago
Some development is fine (and expected). And indeed the goal of "build automation" was what led us to evaluate both Concourse and GitLab. We were previously on Bamboo, which really did require wrangling with a moronic UI. But I think there's a difference between a mostly complete CI product with API extension points, and a "batteries not included" CI toolbox.

jacques_chester 9y ago

> While it has amazing CI pipeline capabilities, we ultimately didn't select Concourse because it felt much more like a CI toolbox, requiring some development to put those tools to use. This is actually a brilliant insight. I'd never thought of it that way. And I'm a one-eyed raving Concourse fan.