53 ms·

Rather than coupling your abstractions to Jenkins/Jenkinsfile/Groovy, I strive to put all commands ran from CI in a `Makefile` or something similar. Ideally, a

Rather than coupling your abstractions to Jenkins/Jenkinsfile/Groovy, I strive to put all commands ran from CI in a `Makefile` or something similar. Ideally, a Jenkins step/stage is a single shell command just with the correct environment variables set. This makes it super easy to reproduce locally, switching to another CI tool is much easier, and your Jenkinsfiles are 20% of the length and only focuses on things directly related to the CI tool itself: parallelization, cleanup, notifications...

xyzzy_plugh 7y ago
You should definitely keep your Jenkins steps around a single command, like executing a Makefile as you suggest. The problem is maintaining the job configuration itself. I've seen some places configure Jenkins jobs for a project in the repo itself, so running a specific type of build depends on it being defined in that revision of the repo you're using.
- peterwwillis 7y ago
  And this is why Jenkins is horrible. You have a Jenkinsfile, which describes jobs in code, but... you need to create a job... to use those other jobs... and there's no simple, officially supported way to do that as code. There are 4 basic ways to manage the initial job configuration: By Hand, Save The XML File, JCasC, and Groovy init files. JCasC is of course the most ideal, but also the least intuitive, and requires further server setup. Putting a Groovy file in an init directory is the most straightforward way, using code. And if the Jenkins server is already running, you need a specific interaction with some API call to actually update the job in the running server. You can also do more horrible things, like Jenkins Job Builder, but I think making someone manage that is against the Geneva Conventions.
  - paulddraper 7y ago
    I haven't used Jenkins in 2 years, but before that I used the Groovy DSL plugin. A job that would run that and create the other jobs. Incidentally, a similar (but native and more streamlined) approach is taken by Buildkite: job that processes a DSL that then generates other jobs (could even be recursive if you like that sort of thing) [1] [1] https://buildkite.com https://buildkite.com
    - peterwwillis 7y ago
      I know that in some cases that's the best solution to a problem, but I try hard to stay away from generative processes. There's a raft of complexity issues that arises from the need to reproduce exactly the same behavior and results, and usually you're introducing several new layers of complexity (a DSL, a job to process it, jobs generated from it, actions taken by those jobs, the input and output of each process... versus one giant static job with minimal variability).
      - paulddraper 7y ago
        It was certainly unpleasantly complicated.
paulddraper 7y ago
That's the right way. Makefile, shell script...something easy. Because then you can also run it locally to produce the same effect. Jenkinsfiles/Groovy DSL/XML files should be the 20 other things: job dependencies, notifications, VCS configuration, retries, coverage reports, etc. Same with anything: Travis, CircleCI, etc.

xyzzy_plugh 7y ago

You should definitely keep your Jenkins steps around a single command, like executing a Makefile as you suggest. The problem is maintaining the job configuration itself. I've seen some places configure Jenkins jobs for a project in the repo itself, so running a specific type of build depends on it being defined in that revision of the repo you're using.

peterwwillis 7y ago
And this is why Jenkins is horrible. You have a Jenkinsfile, which describes jobs in code, but... you need to create a job... to use those other jobs... and there's no simple, officially supported way to do that as code. There are 4 basic ways to manage the initial job configuration: By Hand, Save The XML File, JCasC, and Groovy init files. JCasC is of course the most ideal, but also the least intuitive, and requires further server setup. Putting a Groovy file in an init directory is the most straightforward way, using code. And if the Jenkins server is already running, you need a specific interaction with some API call to actually update the job in the running server. You can also do more horrible things, like Jenkins Job Builder, but I think making someone manage that is against the Geneva Conventions.
- paulddraper 7y ago
  I haven't used Jenkins in 2 years, but before that I used the Groovy DSL plugin. A job that would run that and create the other jobs. Incidentally, a similar (but native and more streamlined) approach is taken by Buildkite: job that processes a DSL that then generates other jobs (could even be recursive if you like that sort of thing) [1] [1] https://buildkite.com https://buildkite.com
  - peterwwillis 7y ago
    I know that in some cases that's the best solution to a problem, but I try hard to stay away from generative processes. There's a raft of complexity issues that arises from the need to reproduce exactly the same behavior and results, and usually you're introducing several new layers of complexity (a DSL, a job to process it, jobs generated from it, actions taken by those jobs, the input and output of each process... versus one giant static job with minimal variability).
    - paulddraper 7y ago
      It was certainly unpleasantly complicated.

paulddraper 7y ago

That's the right way. Makefile, shell script...something easy. Because then you can also run it locally to produce the same effect. Jenkinsfiles/Groovy DSL/XML files should be the 20 other things: job dependencies, notifications, VCS configuration, retries, coverage reports, etc. Same with anything: Travis, CircleCI, etc.