4 ms·

This is highly confusing to me. Pipelines in Jenkins are fragile. You can spend entire days trying to do very common things and run into tons of friction. The d

This is highly confusing to me. Pipelines in Jenkins are fragile. You can spend entire days trying to do very common things and run into tons of friction. The differences between scripted and declarative are severe, in terms of how erros and continuation are handled, the way that plugins interact depending on how you've nested stages and steps. Even things like environment vars and CWD can be a pain to coordinate. Trivial pipelines take far, far, far too long to setup and get working well. This just appears to be a layer on top of the still-limited declarative pipelines. I'm not sure trading a groovy DSL for an extra layer (of yaml, no less) is a good deal. It doesn't solve the weirdness around multibranch, it doesn't address the mismatch between having a parameterized pipeline where that parameterization has to live outside the Jenkinsfile (or if it is encoded in the Jenkinsfile, it acts really oddly, like running the Jenkinsfile actually reconfigures the job that invoked it, etc). This is why everyone has to continue using Groovy DSL and/or JJB to reinstantiate parameterized jobs or handle jobs that deal with multiple Jenkinsfiles in a project. I know Jenkins isn't going anywhere but it's legacy shows a lot.

AnthonBerg 8y ago
Jenkins can stay. I'm going.
- AnthonBerg 8y ago
  Due to a downvote, I want to reaffirm my position as stated.
60secz 8y ago
Don't get me started on stages and concurrency, or trying to figure out where an error came from.
kohsuke 8y ago
First, full disclosure, I'm the creator of Jenkins. I'm sorry to hear that you had a bad experience with Jenkins Pipeline. I can see that you know a lot about it. There's actually no difference between how continuation is handled between scripted and declarative, so I'm curious to know more about what hit you, because I suspect it's something else (though obviously equally frustrating!) Similarly curious about improvement to error reporting, because I think that's something the Pipeline team cares about, and it's one of those things where how errors in real world are made is always more interesting than what we can imagine. I used to work on compilers, so I know the frustration of a poor error message pushing you down a wrong lane, only to discover a few hours later that all it took was one line fix! Modern improvements in Pipeline (like declarative and this one) is in no small part motivated by making those error checks more thorough, easier, and more upfront. So I think this is a change in the right direction. My perception has been that parameterized jobs are in decline, in part because more people are triggering automation implicitly through commits, and not through explicit "run this" button. Parameters are more implied from the context (branch, commit message, creation of tags, etc), as opposed to explicitly given from UI. Stepping back from those specifics, I think regrettably software has bugs, and there's always more usability improvements that can be made, so we are just working on those one at a time, which kinda summarize my entire journey with Jenkins :-) So in that spirit, I want to make sure we learn from your suffering.
- colemickens 8y ago
  Thanks for the response. Mine is a bit terse as I'm on mobile for a few days. For one, the always/finally block in declarative pipelines flatly doesn't work in scripted mode. You have to set/maintain job status yourself and throw/catch. It's very unpleasant. I am shocked to hear that parameterized builds are in decline. Every project I've seen or touched that used Jenkins had multiple pipelines in a repo that needed to be executed with a number of different configurations. If there was better native support for pipelines calling other pipelines, this might not be such a problem. Example: a kube related project. It has a Jenkinsfile for release builds and another for general checkin/PR builds. Each of those need to be tested with two versions of kube, with flannel and calico, with and without RBAC. I don't know of a clean way to do that without Job DSL/JJB and parameterized jobs. In my experience, this type of scenario is not uncommon. We need visibility at the config level. It's not helpful to shove all of that into a single job with coarse -grain reporting up. Even if you put parameters aside, there is just a fundamental mismatch because the pipeline only (today) encodes steps. What about triggers? Having them be part of the pipeline and having the pipeline modify it's own job in Jenkins feels very, very, very wrong to me. There's a reason Job DSL and JJB kept those separate. In fact, if it weren't for the Pipeline visualization, I could've gotten the same functionality with JJB /Job DSL alone, with some bash scripts, with considerably less heartache. I don't know that I know a "lot" about Jenkins, but I do know that I strive for easily repeatable setups that need git triggers, need to post back status, and need to run bash scripts. This most recent attempt was the third time owning it all-up and it was still very tedious. And it was really much easier to achieve this simple requirements with k8s's Prow... Even when having to dive in and hack some of prow's plugins. All of the mature, repeatable Jenkins setups I've seen are small mountains of groovy scripts with their own workarounds embedded. There needs to be a system config DSL too, because maintaining huge blobs of XML and writing groovy.init.d scripts keep me awake at night. Anyway, Jenkins is crazy good stuff, especially for it's age, but for newer smaller projects that don't need the 10,000 Jenkins plugins, it feels unwiedly. I had hope that Jenkins X was going to tackle some of these things, but so far I'm not sure. Thanks again for taking the time, I hope my further feedback is constructive.
  - kohsuke 8y ago

AnthonBerg 8y ago

Jenkins can stay. I'm going.

AnthonBerg 8y ago
Due to a downvote, I want to reaffirm my position as stated.

60secz 8y ago

Don't get me started on stages and concurrency, or trying to figure out where an error came from.

kohsuke 8y ago

First, full disclosure, I'm the creator of Jenkins. I'm sorry to hear that you had a bad experience with Jenkins Pipeline. I can see that you know a lot about it. There's actually no difference between how continuation is handled between scripted and declarative, so I'm curious to know more about what hit you, because I suspect it's something else (though obviously equally frustrating!) Similarly curious about improvement to error reporting, because I think that's something the Pipeline team cares about, and it's one of those things where how errors in real world are made is always more interesting than what we can imagine. I used to work on compilers, so I know the frustration of a poor error message pushing you down a wrong lane, only to discover a few hours later that all it took was one line fix! Modern improvements in Pipeline (like declarative and this one) is in no small part motivated by making those error checks more thorough, easier, and more upfront. So I think this is a change in the right direction. My perception has been that parameterized jobs are in decline, in part because more people are triggering automation implicitly through commits, and not through explicit "run this" button. Parameters are more implied from the context (branch, commit message, creation of tags, etc), as opposed to explicitly given from UI. Stepping back from those specifics, I think regrettably software has bugs, and there's always more usability improvements that can be made, so we are just working on those one at a time, which kinda summarize my entire journey with Jenkins :-) So in that spirit, I want to make sure we learn from your suffering.

colemickens 8y ago
Thanks for the response. Mine is a bit terse as I'm on mobile for a few days. For one, the always/finally block in declarative pipelines flatly doesn't work in scripted mode. You have to set/maintain job status yourself and throw/catch. It's very unpleasant. I am shocked to hear that parameterized builds are in decline. Every project I've seen or touched that used Jenkins had multiple pipelines in a repo that needed to be executed with a number of different configurations. If there was better native support for pipelines calling other pipelines, this might not be such a problem. Example: a kube related project. It has a Jenkinsfile for release builds and another for general checkin/PR builds. Each of those need to be tested with two versions of kube, with flannel and calico, with and without RBAC. I don't know of a clean way to do that without Job DSL/JJB and parameterized jobs. In my experience, this type of scenario is not uncommon. We need visibility at the config level. It's not helpful to shove all of that into a single job with coarse -grain reporting up. Even if you put parameters aside, there is just a fundamental mismatch because the pipeline only (today) encodes steps. What about triggers? Having them be part of the pipeline and having the pipeline modify it's own job in Jenkins feels very, very, very wrong to me. There's a reason Job DSL and JJB kept those separate. In fact, if it weren't for the Pipeline visualization, I could've gotten the same functionality with JJB /Job DSL alone, with some bash scripts, with considerably less heartache. I don't know that I know a "lot" about Jenkins, but I do know that I strive for easily repeatable setups that need git triggers, need to post back status, and need to run bash scripts. This most recent attempt was the third time owning it all-up and it was still very tedious. And it was really much easier to achieve this simple requirements with k8s's Prow... Even when having to dive in and hack some of prow's plugins. All of the mature, repeatable Jenkins setups I've seen are small mountains of groovy scripts with their own workarounds embedded. There needs to be a system config DSL too, because maintaining huge blobs of XML and writing groovy.init.d scripts keep me awake at night. Anyway, Jenkins is crazy good stuff, especially for it's age, but for newer smaller projects that don't need the 10,000 Jenkins plugins, it feels unwiedly. I had hope that Jenkins X was going to tackle some of these things, but so far I'm not sure. Thanks again for taking the time, I hope my further feedback is constructive.
- kohsuke 8y ago