5 ms·

How can we have multiple Jenkinsfiles in a single branch? We want to run one job to build things, then after we deploy, we want to trigger another job that run

How can we have multiple Jenkinsfiles in a single branch? We want to run one job to build things, then after we deploy, we want to trigger another job that runs a bunch of integration, performance, etc. tests against the new code.

hrmpw 10y ago
Why do you need them to be separate Pipelines? Because it is convenient or for other reasons? Do you want everyone of these jobs in every branch that you create? In Pull Requests? Do you want all of these jobs to automatically trigger on every push the repo? It's possible to do all of these steps in a single Pipeline but I'm curious why you would want to separate it?
- crymer11 10y ago
  We want to run our integration tests, etc. against our per-branch staging environment and then against prod when we deploy the branch there as well. With a "post-deploy" pipeline file, we can keep that logic nice and clean and easily trigger the appropriate job via the API. With everything in one file, we would have to add the logic to our Jenkins pipeline to handle when to trigger deployments (which is currently done through a service we built that looks at GitHub commit statuses of a certain prefix/listens to Slack slash commands).
bryanlarsen 10y ago
We have a massive Jenkinsfile with a whole bunch of when clauses so that it can be run in separate cases. Might be nicer as multiple Jenkinsfile's, but it works.
relics443 10y ago
I believe there's a "load" command which takes a path to a jenkinsfile
jsmeaton 10y ago
I was dealing with this just yesterday funny enough. We didn't really like the idea of using `load()` to bring in a new file. I'll prefix the rest by saying we haven't actually implemented this yet, so YMMV. Our goal was to have a separate deploy pipeline that wasn't dependent on the test step (for various reasons I won't go into). We decided on creating a new job of type pipeline, and pointing it at a script named `Jenkinsfile.deploy`. It has a few parameters, namely `BRANCH`, that can be manually set, or passed in from another job. We can then move our deployment steps from the main Jenkinsfile into this new one, and still version control the lot. Major caveat is that the deploy step is not compatible with organisation scans (multibranch/pr), but that's fine for our use case. We usually want to trigger our builds manually based on the branch.
- hrmpw 10y ago
  I was playing around with a combination of Pipeline and Job-DSL and created this as a POC: https://gist.github.com/HRMPW/9b2a3ccbdc370e0a7c9cb541be229da9 https://gist.github.com/HRMPW/9b2a3ccbdc370e0a7c9cb541be229d... This is a Job-DSL seed job written in Declarative Pipeline that pulls in your Organization based on a build parameter and creates two jobs for every repo: 1. A multibranch Pipeline job that is based on the Jenkinsfile that will work across all branches and PRs. 2. A deploy job for the repo that is defined in a "jenkinsfile.deploy" file in the repo. This isn't a multibranch job but can be run on any branch based on a parameter for the branch. This is a very simplistic implementation but could be expanded on further.
- Chico75 10y ago
  Sounds like you would be better off putting the logic in separate function of a global library that you can then call in different stages for different conditions: https://jenkins.io/doc/book/pipeline/shared-libraries/ https://jenkins.io/doc/book/pipeline/shared-libraries/
  - jsmeaton 10y ago
    For some things, yes, and we are planning to do this. Deployment is different between our projects though, so we'd prefer our deployment method to live within the repo beside other build steps.
i386 10y ago
Its possible to include and exclude different stages of the build with any criteria you like using the `when` block. Referring to this example https://gist.github.com/i386/5bacc640574c6d79bb72cd9a1181a505 https://gist.github.com/i386/5bacc640574c6d79bb72cd9a1181a50... if I have a branch called 'feature/cool-feature' then only the stages Build and Test would be run. If the branch is called 'master', all stages in the Pipeline would be run. 'when' also supports environment variables and expressions and even multiple `when` blocks - see the docs at https://jenkins.io/doc/book/pipeline/syntax/#when https://jenkins.io/doc/book/pipeline/syntax/#when

hrmpw 10y ago

Why do you need them to be separate Pipelines? Because it is convenient or for other reasons? Do you want everyone of these jobs in every branch that you create? In Pull Requests? Do you want all of these jobs to automatically trigger on every push the repo? It's possible to do all of these steps in a single Pipeline but I'm curious why you would want to separate it?

crymer11 10y ago
We want to run our integration tests, etc. against our per-branch staging environment and then against prod when we deploy the branch there as well. With a "post-deploy" pipeline file, we can keep that logic nice and clean and easily trigger the appropriate job via the API. With everything in one file, we would have to add the logic to our Jenkins pipeline to handle when to trigger deployments (which is currently done through a service we built that looks at GitHub commit statuses of a certain prefix/listens to Slack slash commands).

bryanlarsen 10y ago

We have a massive Jenkinsfile with a whole bunch of when clauses so that it can be run in separate cases. Might be nicer as multiple Jenkinsfile's, but it works.

relics443 10y ago

I believe there's a "load" command which takes a path to a jenkinsfile

jsmeaton 10y ago

I was dealing with this just yesterday funny enough. We didn't really like the idea of using `load()` to bring in a new file. I'll prefix the rest by saying we haven't actually implemented this yet, so YMMV. Our goal was to have a separate deploy pipeline that wasn't dependent on the test step (for various reasons I won't go into). We decided on creating a new job of type pipeline, and pointing it at a script named `Jenkinsfile.deploy`. It has a few parameters, namely `BRANCH`, that can be manually set, or passed in from another job. We can then move our deployment steps from the main Jenkinsfile into this new one, and still version control the lot. Major caveat is that the deploy step is not compatible with organisation scans (multibranch/pr), but that's fine for our use case. We usually want to trigger our builds manually based on the branch.

hrmpw 10y ago
I was playing around with a combination of Pipeline and Job-DSL and created this as a POC: https://gist.github.com/HRMPW/9b2a3ccbdc370e0a7c9cb541be229da9 https://gist.github.com/HRMPW/9b2a3ccbdc370e0a7c9cb541be229d... This is a Job-DSL seed job written in Declarative Pipeline that pulls in your Organization based on a build parameter and creates two jobs for every repo: 1. A multibranch Pipeline job that is based on the Jenkinsfile that will work across all branches and PRs. 2. A deploy job for the repo that is defined in a "jenkinsfile.deploy" file in the repo. This isn't a multibranch job but can be run on any branch based on a parameter for the branch. This is a very simplistic implementation but could be expanded on further.
Chico75 10y ago
Sounds like you would be better off putting the logic in separate function of a global library that you can then call in different stages for different conditions: https://jenkins.io/doc/book/pipeline/shared-libraries/ https://jenkins.io/doc/book/pipeline/shared-libraries/
- jsmeaton 10y ago
  For some things, yes, and we are planning to do this. Deployment is different between our projects though, so we'd prefer our deployment method to live within the repo beside other build steps.

i386 10y ago

Its possible to include and exclude different stages of the build with any criteria you like using the `when` block. Referring to this example https://gist.github.com/i386/5bacc640574c6d79bb72cd9a1181a505 https://gist.github.com/i386/5bacc640574c6d79bb72cd9a1181a50... if I have a branch called 'feature/cool-feature' then only the stages Build and Test would be run. If the branch is called 'master', all stages in the Pipeline would be run. 'when' also supports environment variables and expressions and even multiple `when` blocks - see the docs at https://jenkins.io/doc/book/pipeline/syntax/#when https://jenkins.io/doc/book/pipeline/syntax/#when