3 ms·

Although Yahoo doesn't seem to be doing very well recently, they've done lots of great engineering. I recently looked through the organization's Github [0] and

Although Yahoo doesn't seem to be doing very well recently, they've done lots of great engineering. I recently looked through the organization's Github [0] and discovered a few nice packages. I'd happily pay for a good managed CD build system capable of handling projects / products with multiple dependencies. Right now I use CircleCI along with a bunch of scripts, and it works, but it's not great. One problem with existing CI tools that I've used is they only handle a single git repo, so extra automation has to be done ad-hoc. It's not terribly difficult, but it's tedious and it doesn't give you any kind of safety guarantees. [0] https://github.com/yahoo https://github.com/yahoo

perlgeek 10y ago
Fwiw https://gocd.io/ https://gocd.io/ handles pipelines (or graphs of multiple piplines) with several git repos just fine. That's on-premise, but not hard to run on your own. There's Snap CI, which is by ThoughtWorks (just as Go CD), and that's in the cloud. I don't know if they use the same code base, but it might be worth investigating if they can deal with multiple repos in a sensible way.
- specialist 10y ago
  We just switched to GoCD. And here I thought Jenkins sucked. I've never missed simple shell scripts, builds triggered by commits, tailing logs so much. Being closer to the metal. Back then, I had no idea the obvious solution was called "continuous integration". GoCD has a pretty dash board though. So I guess that's progress.
  - mariusmg 10y ago
    >So I guess that's progress. No it's not, because scripts are better to debug the problem when (invariably) all goes to shit.
  - edude03 10y ago
    You should try concourse.ci then. I tried Circle gitlab go and concourse and concourse is the best so far.
- bpp 10y ago
  I wasn't responsible for setup, but we've been using Buildkite (https://buildkite.com https://buildkite.com) for our CI and it's been very nice. We run it through a Kubernetes cluster of workers and as far as I can tell it's been very flexible as to what's in a pipeline.
- rdsubhas 10y ago
  There are only 3 ways to model CI: - Plain jobs (disconnected dots) (Jenkins/Travis-CI/etc) - Full-on graphs with value stream maps and arbitrary connections (GoCD/Concourse/etc) - Isolated pipelines (disconnected matrices/lists of columns) (Snap-CI/GitLab CI/etc) I've lamented for a long time that there was no "lightweight" self-hosted pipeline tool that makes full use of modern tech (lightweight API, separate UI, offload the actual build environment to containers) and does nothing more than linking the pipeline and gets out of the way. Most organizations just don't have the engineering skills to handle build graphs. And the alternative job-style ends up in a complete mess. Screwdriver fills a real void. I'm not a fanboy by any chance, but I'm really thankful to Yahoo for opening up this void so that many will follow. Btw, Jenkins 2 Pipelines are not even in the picture because are neither proper pipelines nor proper jobs. It has no resumable/pausable stages, and is just a glorified DSL/UI on top of a multi-job kind of model.
  - vorg 10y ago
    > Btw, Jenkins 2 Pipelines are not even in the picture because are neither proper pipelines nor proper jobs. It has no resumable/pausable stages, and is just a glorified DSL/UI on top of a multi-job kind of model. That glorified DSL is Apache Groovy. Its backers have been trying to distribute it by getting other JVM-based products to bundle it as a DSL, without worrying about how closely it really matches the product's functionality, such as specifying pause/resume in pipelines.

perlgeek 10y ago

Fwiw https://gocd.io/ https://gocd.io/ handles pipelines (or graphs of multiple piplines) with several git repos just fine. That's on-premise, but not hard to run on your own. There's Snap CI, which is by ThoughtWorks (just as Go CD), and that's in the cloud. I don't know if they use the same code base, but it might be worth investigating if they can deal with multiple repos in a sensible way.

specialist 10y ago
We just switched to GoCD. And here I thought Jenkins sucked. I've never missed simple shell scripts, builds triggered by commits, tailing logs so much. Being closer to the metal. Back then, I had no idea the obvious solution was called "continuous integration". GoCD has a pretty dash board though. So I guess that's progress.
- mariusmg 10y ago
  >So I guess that's progress. No it's not, because scripts are better to debug the problem when (invariably) all goes to shit.
- edude03 10y ago
  You should try concourse.ci then. I tried Circle gitlab go and concourse and concourse is the best so far.
bpp 10y ago
I wasn't responsible for setup, but we've been using Buildkite (https://buildkite.com https://buildkite.com) for our CI and it's been very nice. We run it through a Kubernetes cluster of workers and as far as I can tell it's been very flexible as to what's in a pipeline.
rdsubhas 10y ago
There are only 3 ways to model CI: - Plain jobs (disconnected dots) (Jenkins/Travis-CI/etc) - Full-on graphs with value stream maps and arbitrary connections (GoCD/Concourse/etc) - Isolated pipelines (disconnected matrices/lists of columns) (Snap-CI/GitLab CI/etc) I've lamented for a long time that there was no "lightweight" self-hosted pipeline tool that makes full use of modern tech (lightweight API, separate UI, offload the actual build environment to containers) and does nothing more than linking the pipeline and gets out of the way. Most organizations just don't have the engineering skills to handle build graphs. And the alternative job-style ends up in a complete mess. Screwdriver fills a real void. I'm not a fanboy by any chance, but I'm really thankful to Yahoo for opening up this void so that many will follow. Btw, Jenkins 2 Pipelines are not even in the picture because are neither proper pipelines nor proper jobs. It has no resumable/pausable stages, and is just a glorified DSL/UI on top of a multi-job kind of model.
- vorg 10y ago
  > Btw, Jenkins 2 Pipelines are not even in the picture because are neither proper pipelines nor proper jobs. It has no resumable/pausable stages, and is just a glorified DSL/UI on top of a multi-job kind of model. That glorified DSL is Apache Groovy. Its backers have been trying to distribute it by getting other JVM-based products to bundle it as a DSL, without worrying about how closely it really matches the product's functionality, such as specifying pause/resume in pipelines.