4 ms·

Going off of this I wouldn't even use job dsl or pipelines as they allow you to execute arbitrary groovy. I would consider emulating Travis ci or Amazon pipelin

Going off of this I wouldn't even use job dsl or pipelines as they allow you to execute arbitrary groovy. I would consider emulating Travis ci or Amazon pipelines where it is all declarative not imperitive. This way you can re-impliment and consolidate without doing a Turing complete parse of tons of custom groovy scripts

oblio 9y ago
There's a Declarative Pipeline these days plus you can turn on the Groovy sandbox. There's also a YAML based thing but I forgot its name.
- grogenaut 9y ago
  Can you provide links? I'm on a plane so very hard to Google. Seriously? Downvotes? On a holiday on a plane and I get down votes asking for a little help? I did try and google and didn't find the stuff.
  - jeremya 9y ago
    I agree it's not easy to Google for the Declarative Pipeline or Groovy Sandbox. Here's the comparison of the two syntaxes available in Jenkins Pipeline (Scripted and Declarative): https://jenkins.io/doc/book/pipeline/syntax/#compare https://jenkins.io/doc/book/pipeline/syntax/#compare Here are details on the Groovy Sandbox (provided by the Script Security Plugin and installed by default when installing recent Jenkins). The Groovy Sandbox is on by default for Jenkins Pipeline: https://jenkins.io/doc/book/managing/script-approval/ https://jenkins.io/doc/book/managing/script-approval/
    - grogenaut 9y ago
      thanks
  - RyJones 9y ago
    We use JJB, which is YAML based. https://github.com/hyperledger/ci-management/tree/master/jjb https://github.com/hyperledger/ci-management/tree/master/jjb for instance
    - grogenaut 9y ago
      thanks
    - harlowja 9y ago
      Just fyi :) Docs: https://docs.openstack.org/infra/jenkins-job-builder/ https://docs.openstack.org/infra/jenkins-job-builder/ Reviews: https://review.openstack.org/#/q/project:openstack-infra/jenkins-job-builder https://review.openstack.org/#/q/project:openstack-infra/jen... Code: https://git.openstack.org/cgit/openstack-infra/jenkins-job-builder https://git.openstack.org/cgit/openstack-infra/jenkins-job-b...
- jnsaff2 9y ago
  Yea but the declarative pipelines have cheat codes that allow cheeky (read lazy) people to still run code in the pipeline:(
Matt3o12_ 9y ago
I‘m not sure what’s bad about executing arbitrary groovy scripts because you are running arbitrary code anyways (unless Jenkins is not running any code at all – e.g. compile static assets, do something else). If some developer wanted access to the secret files – config files which might have sensitive information such as the database password for the staging environment (I don’t know what they are called in Jenkins) they could just modify the code to dump them to stdout. I am asking this because I don’t see a problem with running turning compete scripts if the application is most likely to run arbitary code anyways, which has access to syscalls any many other dangerous stuff anyways. You need to trust your employees anyways and if you can’t trust them with not dumping the staging environment in purpose, you should not have hired them. It is possible to include harmless looking code which can be malicious and still be considered a bug if found during code review’s.
- grogenaut 9y ago
  The point is that you get mu hich better reuse by not allowing arbitrary groovy and building up interface contracts. This is exactly like any other coding. Using groovy is akin to allowing arbitrary SQL from your web devs intead of apis (from an interface pov not security). Instead just invoke something outside of Jenkins groovy framework like maven, make, etc. Makes you much more portable and upgradable. Jenkins should just be your execution/workflow environment not a dev framework. Also never mutate the worker boxes, everything should be completely isloated from itself and each other. Jenkins is like any other software architecture. Finally doing too much in Jenkins makes it impossible to test Jenkins locally, which means you can neither debug builds locally without pushing new code to build, nor can you test alterations to Jenkins itself locally which makes it much much more fragile. How do I know? I'm a sr dev coming in and fixing a mid sized engineering org (400+ devs) jenkins 4 years of accretions. These are major issues we've identified in just a week of looking. Build systems should be simple, scalaboe, agile, and composable, not monolithic. Keep the monolithic to the individual builds not cross company.
  - Matt3o12_ 9y ago
    Got it, my groovy files have always been setup the environment (I.e. write config files for the test database, API keys, etc everything that should not be in git), and then just execute make to make so that I don’t have a completely different setup each environment. Thank you very much!
vorg 9y ago
> Travis ci or Amazon pipelines where it is all declarative not imperitive [...] a Turing complete parse of tons of custom groovy scripts Even worse than Apache Groovy being an imperative DSL (compared to Travis ci and Amazon pipelines being all declarative) is that the most declarative portions of Groovy (i.e. its collections API) are disabled when used in the Jenkins pipelines.

oblio 9y ago

There's a Declarative Pipeline these days plus you can turn on the Groovy sandbox. There's also a YAML based thing but I forgot its name.

grogenaut 9y ago
Can you provide links? I'm on a plane so very hard to Google. Seriously? Downvotes? On a holiday on a plane and I get down votes asking for a little help? I did try and google and didn't find the stuff.
- jeremya 9y ago
  I agree it's not easy to Google for the Declarative Pipeline or Groovy Sandbox. Here's the comparison of the two syntaxes available in Jenkins Pipeline (Scripted and Declarative): https://jenkins.io/doc/book/pipeline/syntax/#compare https://jenkins.io/doc/book/pipeline/syntax/#compare Here are details on the Groovy Sandbox (provided by the Script Security Plugin and installed by default when installing recent Jenkins). The Groovy Sandbox is on by default for Jenkins Pipeline: https://jenkins.io/doc/book/managing/script-approval/ https://jenkins.io/doc/book/managing/script-approval/
  - grogenaut 9y ago
    thanks
- RyJones 9y ago
  We use JJB, which is YAML based. https://github.com/hyperledger/ci-management/tree/master/jjb https://github.com/hyperledger/ci-management/tree/master/jjb for instance
  - grogenaut 9y ago
    thanks
  - harlowja 9y ago
    Just fyi :) Docs: https://docs.openstack.org/infra/jenkins-job-builder/ https://docs.openstack.org/infra/jenkins-job-builder/ Reviews: https://review.openstack.org/#/q/project:openstack-infra/jenkins-job-builder https://review.openstack.org/#/q/project:openstack-infra/jen... Code: https://git.openstack.org/cgit/openstack-infra/jenkins-job-builder https://git.openstack.org/cgit/openstack-infra/jenkins-job-b...
jnsaff2 9y ago
Yea but the declarative pipelines have cheat codes that allow cheeky (read lazy) people to still run code in the pipeline:(

Matt3o12_ 9y ago

I‘m not sure what’s bad about executing arbitrary groovy scripts because you are running arbitrary code anyways (unless Jenkins is not running any code at all – e.g. compile static assets, do something else). If some developer wanted access to the secret files – config files which might have sensitive information such as the database password for the staging environment (I don’t know what they are called in Jenkins) they could just modify the code to dump them to stdout. I am asking this because I don’t see a problem with running turning compete scripts if the application is most likely to run arbitary code anyways, which has access to syscalls any many other dangerous stuff anyways. You need to trust your employees anyways and if you can’t trust them with not dumping the staging environment in purpose, you should not have hired them. It is possible to include harmless looking code which can be malicious and still be considered a bug if found during code review’s.

grogenaut 9y ago
The point is that you get mu hich better reuse by not allowing arbitrary groovy and building up interface contracts. This is exactly like any other coding. Using groovy is akin to allowing arbitrary SQL from your web devs intead of apis (from an interface pov not security). Instead just invoke something outside of Jenkins groovy framework like maven, make, etc. Makes you much more portable and upgradable. Jenkins should just be your execution/workflow environment not a dev framework. Also never mutate the worker boxes, everything should be completely isloated from itself and each other. Jenkins is like any other software architecture. Finally doing too much in Jenkins makes it impossible to test Jenkins locally, which means you can neither debug builds locally without pushing new code to build, nor can you test alterations to Jenkins itself locally which makes it much much more fragile. How do I know? I'm a sr dev coming in and fixing a mid sized engineering org (400+ devs) jenkins 4 years of accretions. These are major issues we've identified in just a week of looking. Build systems should be simple, scalaboe, agile, and composable, not monolithic. Keep the monolithic to the individual builds not cross company.
- Matt3o12_ 9y ago
  Got it, my groovy files have always been setup the environment (I.e. write config files for the test database, API keys, etc everything that should not be in git), and then just execute make to make so that I don’t have a completely different setup each environment. Thank you very much!

vorg 9y ago

> Travis ci or Amazon pipelines where it is all declarative not imperitive [...] a Turing complete parse of tons of custom groovy scripts Even worse than Apache Groovy being an imperative DSL (compared to Travis ci and Amazon pipelines being all declarative) is that the most declarative portions of Groovy (i.e. its collections API) are disabled when used in the Jenkins pipelines.