5 ms·

It's cool that they're promoting the "pipeline" plugin to a built-in feature, but the devil is in the details. Under the hood, it's implemented by taking a scr

It's cool that they're promoting the "pipeline" plugin to a built-in feature, but the devil is in the details. Under the hood, it's implemented by taking a script written in a "DSL" (it's actually Groovy code), transforming it into continuation-passing style, and serializing its state. This is pretty cool from a theoretical CS perspective, but having played with it a little bit, the implementation seems very fragile. There are a number of long-standing unfixed bugs that cause the transformed code to misbehave in ways that you wouldn't predict just from looking at it, even if you're experienced with Groovy. I ran into a couple of them just during a brief period of experimentation. For instance: https://issues.jenkins-ci.org/browse/JENKINS-27893 https://issues.jenkins-ci.org/browse/JENKINS-27893 (varargs are mishandled) https://issues.jenkins-ci.org/browse/JENKINS-28277 https://issues.jenkins-ci.org/browse/JENKINS-28277 (currying doesn't work) https://issues.jenkins-ci.org/browse/JENKINS-26481 https://issues.jenkins-ci.org/browse/JENKINS-26481 (calling a closure in a loop doesn't work right) https://issues.jenkins-ci.org/browse/JENKINS-28183 https://issues.jenkins-ci.org/browse/JENKINS-28183 (killing a pipeline job doesn't clean up its state properly) And there are inherent limitations to the approach; for instance, you can't store any non-serializable values in local variables, which means simple things like foreach loops don't work (because you can't serialize an iterator). I really like the idea behind the Pipelines feature, in principle. But I think building it around a general-purpose programming language, and then failing to support all of that language's features, is a recipe for headaches. If you never try to do anything more complicated than what's shown in the examples then you should be fine, but the borders between what works and what doesn't are very ill-defined. EDIT: Oh yeah, and there's a surprising amount of functionality that isn't documented anywhere except for blog posts and JIRA tickets.

vorg 11y ago
> Under the hood, it's implemented by taking a script written in a "DSL" (it's actually Groovy code) It's Groovy code? The announcement page doesn't mention "Groovy" anywhere, let alone "Apache Groovy" which is what the backers of Groovy are meant to refer to it as since last November (2015) when it was promoted from the Apache incubator by the ASF.
- teraflop 11y ago
  Yep. You'd think this kind of thing would be considered significant enough to mention, but apparently not. They do mention it in the "tutorial", which AFAICT constitutes the entirety of the official documentation: https://github.com/jenkinsci/workflow-plugin/blob/master/TUTORIAL.md#understanding-pipeline-scripts https://github.com/jenkinsci/workflow-plugin/blob/master/TUT... > A pipeline is a Groovy script that tells Jenkins what to do when your Pipeline is run.
incepted 11y ago
DSL are certainly the future of this kind of tools but not with a dynamically typed language like Groovy. The fact that it's dynamically typed makes it pretty much impossible to get decent support from IDE's. Gradle's support in IDEA has been broken forever, and you know that even if the JetBrains guys can't pull it off, it can't be pulled off. I think the future belongs to statically typed languages that can offer DSL's as terse as Groovy but with actually types in them, so that the IDE can actually do the job it's being paid for.
- pron 11y ago
  > I think the future belongs to statically typed languages that can offer DSL's as terse as Groovy but with actually types in them, so that the IDE can actually do the job it's being paid for. Not as easy as it sounds. If the DSL is rich enough, and if the types are handled by the host language, type errors can become just as cryptic and unhelpful as with an untyped DSLs, and it's just as hard for the IDE to do something meaningful with them.
  - incepted 11y ago
    Sure, but at least, auto completion works automatically. With dynamically typed languages, you get cryptic (and vague) errors and auto completion just can't work.
    - icebraining 11y ago
      I use auto-completion in Python every day with PyDev. It's not as advanced as for other languages, but "just can't work" is not really true either. http://www.pydev.org/manual_adv_complctx.html http://www.pydev.org/manual_adv_complctx.html
    - pron 11y ago
      That also depends on the namespaces used by the language. E.g. auto-completion in Clojure works a lot better than in JS, because every symbol has a unique resolution, known statically (or as unique as in many typed languages). True, you get less helpful suggestions, but OTOH, the error messages can be much clearer than in the typed case. The former can leave error reporting to the DSL, while the latter is restricted to the cryptic types of the host language's compiler. I'm not saying this can't be resolved with some clever compiler plugins, but if we're comparing options available now, the advantage is not so clear-cut (I do favor types, but only as long as they don't get too complex).
empath75 11y ago
the broken for loop functionally wasted an entire day for me because I didn't know if the problem was me or groovy or Jenkins, being unfamiliar with groovy.

vorg 11y ago

> Under the hood, it's implemented by taking a script written in a "DSL" (it's actually Groovy code) It's Groovy code? The announcement page doesn't mention "Groovy" anywhere, let alone "Apache Groovy" which is what the backers of Groovy are meant to refer to it as since last November (2015) when it was promoted from the Apache incubator by the ASF.

teraflop 11y ago
Yep. You'd think this kind of thing would be considered significant enough to mention, but apparently not. They do mention it in the "tutorial", which AFAICT constitutes the entirety of the official documentation: https://github.com/jenkinsci/workflow-plugin/blob/master/TUTORIAL.md#understanding-pipeline-scripts https://github.com/jenkinsci/workflow-plugin/blob/master/TUT... > A pipeline is a Groovy script that tells Jenkins what to do when your Pipeline is run.

incepted 11y ago

DSL are certainly the future of this kind of tools but not with a dynamically typed language like Groovy. The fact that it's dynamically typed makes it pretty much impossible to get decent support from IDE's. Gradle's support in IDEA has been broken forever, and you know that even if the JetBrains guys can't pull it off, it can't be pulled off. I think the future belongs to statically typed languages that can offer DSL's as terse as Groovy but with actually types in them, so that the IDE can actually do the job it's being paid for.

pron 11y ago
> I think the future belongs to statically typed languages that can offer DSL's as terse as Groovy but with actually types in them, so that the IDE can actually do the job it's being paid for. Not as easy as it sounds. If the DSL is rich enough, and if the types are handled by the host language, type errors can become just as cryptic and unhelpful as with an untyped DSLs, and it's just as hard for the IDE to do something meaningful with them.
- incepted 11y ago
  Sure, but at least, auto completion works automatically. With dynamically typed languages, you get cryptic (and vague) errors and auto completion just can't work.
  - icebraining 11y ago
    I use auto-completion in Python every day with PyDev. It's not as advanced as for other languages, but "just can't work" is not really true either. http://www.pydev.org/manual_adv_complctx.html http://www.pydev.org/manual_adv_complctx.html
  - pron 11y ago
    That also depends on the namespaces used by the language. E.g. auto-completion in Clojure works a lot better than in JS, because every symbol has a unique resolution, known statically (or as unique as in many typed languages). True, you get less helpful suggestions, but OTOH, the error messages can be much clearer than in the typed case. The former can leave error reporting to the DSL, while the latter is restricted to the cryptic types of the host language's compiler. I'm not saying this can't be resolved with some clever compiler plugins, but if we're comparing options available now, the advantage is not so clear-cut (I do favor types, but only as long as they don't get too complex).

empath75 11y ago

the broken for loop functionally wasted an entire day for me because I didn't know if the problem was me or groovy or Jenkins, being unfamiliar with groovy.