3 ms·

If we were hypothetically "too big" for TeamCity, with scaling difficulties all over, do you have (or have heard) any recommendations? I hear the biggest compa

by nod 11y ago

If we were hypothetically "too big" for TeamCity, with scaling difficulties all over, do you have (or have heard) any recommendations? I hear the biggest companies all roll their own, but is there anything between TeamCity and Google-scale?

wmfiv 11y ago
What is too big? Too many concurrent jobs? We found the integration with AWS to autolaunch workers to be quite easy to setup.
dominotw 11y ago
We use jenkins at medium scale. > 400 jobs, 20k-30k builds/day. Works just fine.
- seanp2k2 11y ago
  Jenkins works fine with dozens to hundreds of slaves per master, thousands of jobs, and tens of thousands of builds per day. Jenkins isn't perfect for everything, but it can be a very useful task runner. People have adapted it to do almost everything these days, from database maintenances to builds to restarting infrastructure. Given the almost unlimited triggering and reporting, you could use it to build many ETL apps as well (not saying that's a good idea). As much as Jenkins sucks sometimes, it's what many big shops use for many things. There is tons of community around it. It works, it's battle-tested, and it's been that way for years. Anything else will have a hard time catching up for as many use-cases as Jenkins currently solves for people. If you hate a part of it, fix it and open a PR :)
stormbeta 11y ago
Jenkins + generated jobs (i.e. jenkins-job-dsl or the new Pipeline DSL), and keep as much as the build configuration and logic in your project where it belongs. The configuration that lives on the CI server should be pretty minimal. We have a custom wrapper around the jenkins-job-dsl that encodes our conventions and templates to generate several hundred jobs, and it works very well for our medium-scale instance.
zabilcm 11y ago
There's http://go.cd http://go.cd if you are looking at something focused on CD (Pipelines etc)
- fathom108 11y ago
  Yes, we've found gocd does a terrific job of allowing you cleanly model your full continuous delivery workflow and organize your servers.

by nod 11y ago

wmfiv 11y ago

What is too big? Too many concurrent jobs? We found the integration with AWS to autolaunch workers to be quite easy to setup.

dominotw 11y ago

We use jenkins at medium scale. > 400 jobs, 20k-30k builds/day. Works just fine.

seanp2k2 11y ago
Jenkins works fine with dozens to hundreds of slaves per master, thousands of jobs, and tens of thousands of builds per day. Jenkins isn't perfect for everything, but it can be a very useful task runner. People have adapted it to do almost everything these days, from database maintenances to builds to restarting infrastructure. Given the almost unlimited triggering and reporting, you could use it to build many ETL apps as well (not saying that's a good idea). As much as Jenkins sucks sometimes, it's what many big shops use for many things. There is tons of community around it. It works, it's battle-tested, and it's been that way for years. Anything else will have a hard time catching up for as many use-cases as Jenkins currently solves for people. If you hate a part of it, fix it and open a PR :)

stormbeta 11y ago

Jenkins + generated jobs (i.e. jenkins-job-dsl or the new Pipeline DSL), and keep as much as the build configuration and logic in your project where it belongs. The configuration that lives on the CI server should be pretty minimal. We have a custom wrapper around the jenkins-job-dsl that encodes our conventions and templates to generate several hundred jobs, and it works very well for our medium-scale instance.

zabilcm 11y ago

There's http://go.cd http://go.cd if you are looking at something focused on CD (Pipelines etc)

fathom108 11y ago
Yes, we've found gocd does a terrific job of allowing you cleanly model your full continuous delivery workflow and organize your servers.