6 ms·

They should rethink the deployment model instead. You cannot provision jenkins unattended without 3rd party hacks and undocumented features. Until this is fix

They should rethink the deployment model instead. You cannot provision jenkins unattended without 3rd party hacks and undocumented features. Until this is fixed I recommend avoiding it, as you'll get pet servers. This is totally retrograde to the devops mindset. Now why should I not use that mindset if that proves to be productive at other parts of work?

oblio 10y ago
I'm not sure I get the "provision Jenkins part". Slaves "self install" once you have a working SSH connection and the master is just a bunch of XML files.
- kodfodrasz 10y ago
  Have you tried setting up the jenkins docker image? I'd expect an experience where it works out of the box, or works out of the box after setting up some environment variables. Actual experience: you need to manually click a wizard, and set up an admin password, or google and find a hidden github issue of an ansible playbook with a workaround how to make jenkins startable for the first time unattended. Just a bunch of xmls? Which ones? With what content? Have you tried ansible for example? I'd like to declaratively define my initial setup: - jenkins admin password, users, passwords, roles, ssh keys - slaves - jenkins plugins can be listed and are installed - git repository lists which contain the Jenkinsfiles Good luck doing this non-iteractively with those bunch of XML files... (the jenkins cli randomly does not support parts of these steps)
  - ek750 10y ago
    Not in docker, but I've setup a Jenkins cluster up using chef. I agree it should be easier, but it is possible today.
Terretta 10y ago
I'd be very interested in i386's comments here.
- i386 10y ago
  TBH there's not much I can say on this as its not my particular area of focus (I'm the "bring CD to everyone" kinda guy) but I and other community members do recognise it is a problem that we want to solve. With Pipeline there is a trend to moving job configuration to source control and I think that direction will make things more Ops friendly as time goes by.
  - yawgmoth 10y ago
    I wouldn't call it just a trend, it's part and parcel with the DevOps philosophy. Infrastructure as code, in source control. It's more of a conclusion, the trend is adoption.
  - sdiepend 10y ago
    Not only job configuration. But the entire configuration of your jenkins setup. For example this issue was a blocking factor for automatically provisioning a new jenkins. https://issues.jenkins-ci.org/browse/JENKINS-34035 https://issues.jenkins-ci.org/browse/JENKINS-34035
haldean 10y ago
You can "apt-get install jenkins" and go straight to the webui. Or are you talking about restoring a configuration as well?
- deleted 10y ago
  [deleted]
- mtalantikite 10y ago
  I'd guess the OP means restoring configs and the fact that Jenkins setups tend to turn into snowflake servers. Here are some issues: - State is stored on the filesystem across a bunch of XML files, rather than in a DB. - Configuration of Jenkins itself happens primarily from the UI rather than in config files. This makes it very difficult to provision automatically. - No HA story. Or there is a workaround and it relies on NFS. Which is basically the same as no HA story. For a piece of critical infrastructure this is very bad. - Startup time is horrendous. If you build an AMI and put it in an autoscaling group to attempt to get some vague approximation of HA, boot times often tend to be so long that you need to increase your warmup health check a ton so the autoscaling group doesn't kill the server in an infinite loop. Sure, you can ssh into your server and apt-get install it, but really you should be writing automation and not doing setups by hand. So while it's nice to see some effort focused on the UI, since Jenkins relies on it almost exclusively, there are a lot of other fundamental problems. I always try to use anything other than Jenkins unless forced to.
  - corney91 10y ago
    I agree Jenkins isn't the easiest to automate but surely the fact that it's all XML files makes it easier than a database -- using config management you can install Jenkins and template out the necessary configuration file(s). I much prefer systems that use files instead of the db for configuration...
    - mtalantikite 10y ago
      Configs should be in config files, state in a database. They are coupled in Jenkins and both are on the filesystem, mainly in XML files that are undocumented. Sorry if I wasn't clear about that.
dominotw 10y ago
>You cannot provision jenkins unattended What do you mean by this ? what needs attending?
- falsedan 10y ago
  Setting up Global Security Settings i.e. configuring LDAP creds
karlmdavis 10y ago
FWIW, if you fight through the "undocumented features," Jenkins actually ends up being one of the best tools out there to manage via Ansible or whatnot. The Groovy/Java API for Jenkins is incredibly powerful, and by adding Groovy scripts to the /var/lib/jenkins/init.groovy.d/ directory, you can configure just about anything. Maybe this will be helpful? Here's the Ansible config I've built for a Jenkins deployment I manage: https://github.com/HHSIDEAlab/hhsdevcloud-ansible/tree/master/roles/jenkins https://github.com/HHSIDEAlab/hhsdevcloud-ansible/tree/maste.... Most of the "meat" is in the Groovy scripts in the templates/ directory there.
- vorg 10y ago
  Apache Groovy also has "undocumented features", e.g. from your scripts: matchingSecret = secrets.find { secret -> secret.id == id } can be written as: matchingSecret = secrets.find { it.id == id } Those scripts look fairly basic, without any looping. Perhaps Java 8 with lambdas, or Kotlin could be used instead.
  - dmux 10y ago
    For more about this, see the documentation relating to the "implicit parameter"[0] [0] http://groovy-lang.org/closures.html#implicit-it http://groovy-lang.org/closures.html#implicit-it
kohsuke 10y ago
If I understand you correctly, you are talking about managing Jenkins configuration as code. For that, see * configure jobs as code (https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin) * system configuration as code (https://issues.jenkins-ci.org/browse/JENKINS-31094 https://issues.jenkins-ci.org/browse/JENKINS-31094) and its current prototype (https://github.com/jenkinsci/system-config-dsl-plugin https://github.com/jenkinsci/system-config-dsl-plugin) * bypassing setup wizard (https://issues.jenkins-ci.org/browse/JENKINS-34035 https://issues.jenkins-ci.org/browse/JENKINS-34035) There are also puppet modules, Chef cookbooks, and etc that a lot of people use. Some of those are maintained by people who are also in the Jenkins community, so I don't consider them "3rd party hacks" if you are referring to those.
- deleted 10y ago
  [deleted]
- mtalantikite 10y ago
  And this is one of the major shortfalls of the Jenkins mindset. Everything is a plugin and an afterthought from the main design of the system. Something as crucial as configuring the system should be a top design priority, not shoved into a plugin 5 years after the launch of the project. So, great, there is a plugin to deal with configuration. How do I bootstrap my infrastructure? If it's a plugin, it sounds like I need to install that in an already running Jenkins setup, no? Currently, both this and the bypassing setup wizard seem like hacks to me. Configuration of a critical piece of your infrastructure should never be a hack.
  - oblio 10y ago
    Jenkins is more than 10 years old :) Regarding plugins, you can just plop them in the plugins folder an start Jenkins.
    - deleted 10y ago
      [deleted]
    - 10y ago
flurdy 10y ago
Im not a Jenkins fan. But at my last client I did appreciate that all our jobs where under source control and jenkins could be reprovisioned at any time from source control by using: https://github.com/hmrc/jenkins-job-builders https://github.com/hmrc/jenkins-job-builders and https://github.com/hmrc/jenkins-jobs https://github.com/hmrc/jenkins-jobs Bonus is that you did not have to use the awful Jenkins UI. Though Blue Ocean looks interesting

oblio 10y ago

I'm not sure I get the "provision Jenkins part". Slaves "self install" once you have a working SSH connection and the master is just a bunch of XML files.

kodfodrasz 10y ago
Have you tried setting up the jenkins docker image? I'd expect an experience where it works out of the box, or works out of the box after setting up some environment variables. Actual experience: you need to manually click a wizard, and set up an admin password, or google and find a hidden github issue of an ansible playbook with a workaround how to make jenkins startable for the first time unattended. Just a bunch of xmls? Which ones? With what content? Have you tried ansible for example? I'd like to declaratively define my initial setup: - jenkins admin password, users, passwords, roles, ssh keys - slaves - jenkins plugins can be listed and are installed - git repository lists which contain the Jenkinsfiles Good luck doing this non-iteractively with those bunch of XML files... (the jenkins cli randomly does not support parts of these steps)
- ek750 10y ago
  Not in docker, but I've setup a Jenkins cluster up using chef. I agree it should be easier, but it is possible today.

Terretta 10y ago

I'd be very interested in i386's comments here.

i386 10y ago
TBH there's not much I can say on this as its not my particular area of focus (I'm the "bring CD to everyone" kinda guy) but I and other community members do recognise it is a problem that we want to solve. With Pipeline there is a trend to moving job configuration to source control and I think that direction will make things more Ops friendly as time goes by.
- yawgmoth 10y ago
  I wouldn't call it just a trend, it's part and parcel with the DevOps philosophy. Infrastructure as code, in source control. It's more of a conclusion, the trend is adoption.
- sdiepend 10y ago
  Not only job configuration. But the entire configuration of your jenkins setup. For example this issue was a blocking factor for automatically provisioning a new jenkins. https://issues.jenkins-ci.org/browse/JENKINS-34035 https://issues.jenkins-ci.org/browse/JENKINS-34035

haldean 10y ago

You can "apt-get install jenkins" and go straight to the webui. Or are you talking about restoring a configuration as well?

deleted 10y ago
[deleted]
mtalantikite 10y ago
I'd guess the OP means restoring configs and the fact that Jenkins setups tend to turn into snowflake servers. Here are some issues: - State is stored on the filesystem across a bunch of XML files, rather than in a DB. - Configuration of Jenkins itself happens primarily from the UI rather than in config files. This makes it very difficult to provision automatically. - No HA story. Or there is a workaround and it relies on NFS. Which is basically the same as no HA story. For a piece of critical infrastructure this is very bad. - Startup time is horrendous. If you build an AMI and put it in an autoscaling group to attempt to get some vague approximation of HA, boot times often tend to be so long that you need to increase your warmup health check a ton so the autoscaling group doesn't kill the server in an infinite loop. Sure, you can ssh into your server and apt-get install it, but really you should be writing automation and not doing setups by hand. So while it's nice to see some effort focused on the UI, since Jenkins relies on it almost exclusively, there are a lot of other fundamental problems. I always try to use anything other than Jenkins unless forced to.
- corney91 10y ago
  I agree Jenkins isn't the easiest to automate but surely the fact that it's all XML files makes it easier than a database -- using config management you can install Jenkins and template out the necessary configuration file(s). I much prefer systems that use files instead of the db for configuration...
  - mtalantikite 10y ago
    Configs should be in config files, state in a database. They are coupled in Jenkins and both are on the filesystem, mainly in XML files that are undocumented. Sorry if I wasn't clear about that.

dominotw 10y ago

>You cannot provision jenkins unattended What do you mean by this ? what needs attending?

falsedan 10y ago
Setting up Global Security Settings i.e. configuring LDAP creds

karlmdavis 10y ago

FWIW, if you fight through the "undocumented features," Jenkins actually ends up being one of the best tools out there to manage via Ansible or whatnot. The Groovy/Java API for Jenkins is incredibly powerful, and by adding Groovy scripts to the /var/lib/jenkins/init.groovy.d/ directory, you can configure just about anything. Maybe this will be helpful? Here's the Ansible config I've built for a Jenkins deployment I manage: https://github.com/HHSIDEAlab/hhsdevcloud-ansible/tree/master/roles/jenkins https://github.com/HHSIDEAlab/hhsdevcloud-ansible/tree/maste.... Most of the "meat" is in the Groovy scripts in the templates/ directory there.

vorg 10y ago
Apache Groovy also has "undocumented features", e.g. from your scripts: matchingSecret = secrets.find { secret -> secret.id == id } can be written as: matchingSecret = secrets.find { it.id == id } Those scripts look fairly basic, without any looping. Perhaps Java 8 with lambdas, or Kotlin could be used instead.
- dmux 10y ago
  For more about this, see the documentation relating to the "implicit parameter"[0] [0] http://groovy-lang.org/closures.html#implicit-it http://groovy-lang.org/closures.html#implicit-it

kohsuke 10y ago

If I understand you correctly, you are talking about managing Jenkins configuration as code. For that, see * configure jobs as code (https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin) * system configuration as code (https://issues.jenkins-ci.org/browse/JENKINS-31094 https://issues.jenkins-ci.org/browse/JENKINS-31094) and its current prototype (https://github.com/jenkinsci/system-config-dsl-plugin https://github.com/jenkinsci/system-config-dsl-plugin) * bypassing setup wizard (https://issues.jenkins-ci.org/browse/JENKINS-34035 https://issues.jenkins-ci.org/browse/JENKINS-34035) There are also puppet modules, Chef cookbooks, and etc that a lot of people use. Some of those are maintained by people who are also in the Jenkins community, so I don't consider them "3rd party hacks" if you are referring to those.

deleted 10y ago
[deleted]
mtalantikite 10y ago
And this is one of the major shortfalls of the Jenkins mindset. Everything is a plugin and an afterthought from the main design of the system. Something as crucial as configuring the system should be a top design priority, not shoved into a plugin 5 years after the launch of the project. So, great, there is a plugin to deal with configuration. How do I bootstrap my infrastructure? If it's a plugin, it sounds like I need to install that in an already running Jenkins setup, no? Currently, both this and the bypassing setup wizard seem like hacks to me. Configuration of a critical piece of your infrastructure should never be a hack.
- oblio 10y ago
  Jenkins is more than 10 years old :) Regarding plugins, you can just plop them in the plugins folder an start Jenkins.
  - deleted 10y ago
    [deleted]
  - 10y ago

flurdy 10y ago

Im not a Jenkins fan. But at my last client I did appreciate that all our jobs where under source control and jenkins could be reprovisioned at any time from source control by using: https://github.com/hmrc/jenkins-job-builders https://github.com/hmrc/jenkins-job-builders and https://github.com/hmrc/jenkins-jobs https://github.com/hmrc/jenkins-jobs Bonus is that you did not have to use the awful Jenkins UI. Though Blue Ocean looks interesting