7 ms·

Since everybody seems to be hating on jenkins so much, I'll speak up. IMO jenkins is one of the most valuable tools at any startup and every good engineer uses

Since everybody seems to be hating on jenkins so much, I'll speak up. IMO jenkins is one of the most valuable tools at any startup and every good engineer uses either jenkins or something very similar. There's no comparison between managed/logged/permissioned/distributed jobs that jenkins provides for free vs building an overwrought service or an insufficient crontab. However it's a power-tool and I think a lot of people go in expecting something dead-simple and pretty and are put off by something that they need to invest in learning. Just for example, in 2 days I built an uptime checker that ran every 60-seconds via jenkins and triggered slack alarms/sms on problem, and it just worked (< 2 days total maintenance), for years. An equivalent service (Pingdom) quoted 10k a year.

munk-a 5y ago
I think Jenkins is a very powerful tool but I would strongly disagree that it's quite valuable in a startup environment. Jenkins has struck me as a philosophical sibling as C++ - it offers a plethora of foot guns to the user that can also be used to accomplish rather good things. I would advise Jenkins to companies with projects that don't require agile responses to user requirements where the functionality desired at the end is very clear from the beginning - since the times I see Jenkins fail the most are when the configuration of jobs on it are being updated and reconfigured rapidly. Jenkins tends to suffer from extremely poor state management and experimenting with things can cause a permanent loss of value to companies if backups are not properly configured - compared with a system like ansible where recipes will grow and be regularly committed to a repository Jenkins doesn't have a nature version control system and as a result strikes me as extremely brittle. I almost wish the UI for jenkins simply didn't exist and it actually was just a whole mess of config files and shell scripts so that it could be locked down to a much better degree.
- zug_zug 5y ago
  I don't care to unpack most of that. But what I will say is that I recommend automatically backup up jenkins nightly, I accomplished this simply by commiting jenkins_home to a git repo and pushing every night.
  - oblio 5y ago
    These days you can also use the Configuration as Code plugin and you can automate the setup, version control it, document it, etc: https://www.jenkins.io/projects/jcasc/ https://www.jenkins.io/projects/jcasc/
  - 2rsf 5y ago
    But did you try to restore from backup? I did and was flawless and smooth
aranelsurion 5y ago
Seems to me people hating on Jenkins are not hating on the idea of a central automation service/task runner, they hate on Jenkins itself.
jordanbeiber 5y ago
You could use a tool desgned for it and probably be better of, since upgrading jenkins and plugins, as everyone knows... is not always exactly pain free. Regarding your batch scheduling solution - take hashicorp nomad for a spin: a single go binary and your scheduled job can be declared in 10 lines of yaml. Won’t miss a beat.
- TomBombadildoze 5y ago
  This isn't a batch scheduling problem. It's a monitoring and alerting problem. OP needs a monitoring and alerting solution. Stop ~~~ engineering ~~~ things and choose the right tool for the job.
  - mixologic 5y ago
    "Solution" and "Job" may be overstating things here. If all op needs is monitoring and alerting of a couple of machines, then Datadog/New Relic/Ops Genie/Pager Duty/Server Density/Pingdom/AppDynamics/Loupe/Sysdig/Dynatrace (just to name a few in the crowded space) are all likely overkill and not worth the cost. A large portion of the cost of many of these tools is spent "choosing the right tool for the job" Figuring out what they do, what they do well, where they overlap, where the company that makes the tool is headed, how hard its going to be to swap that tool out for a better one (or cheaper one) - thats a lot of expense in labor and training.
  - jordanbeiber 5y ago
    A script scheduled to test something and notify? Hardly ”engineered”. I claim using pretty much any tool other than jenkins is easier to maintain over time (as illustrated in this very article).
codeduck 5y ago
I've been using jenkins for over 10 years. There is no finer system for turning quick CI hacks into entrenched technical debt. Jenkins gives you the freedom to do things without the burden of asking whether you should do those things. Jenkins permits you to munge a whole lot of separate domains into a single, creaky system. There are better tools available for most aspects of CICD and automated testing these days.
- dijit 5y ago
  > There are better tools available for most aspects of CICD and automated testing these days. It's easy to say this and not cite examples. I rock perforce and windows agents, requirements of the environment, and we struggled very hard to find something that beats Jenkins. One guy even bought an Octopus Deploy license for a year (at the tune of a few thousand dollary-doos) and it did everything worse than our Jenkins setup. I actually dislike Jenkins, I'm open to suggestions here.
  - devonkim 5y ago
    Jenkins to me is great for prototyping certain process activities in a very arbitrary set of conventions, but it provides a lesson as to why the Java world moved on from tools like Apache Ant (XML-based Make) to Maven (convention over configuration as a tooling value). While 10 years ago such processes were not really standardized nowadays people are used to conventions like GitOps with well documented processes for how artifacts are built, deployed, and operated with container primitives after we've spent a _lot_ of time now trying to manage Jenkins installations that become incredibly brittle over time and become more of a liability than a force multiplier for the organization. So people are adopting more specialized tooling instead of trying to have Jenkins do everything under the sun from building artifacts to deploying them to deploying infrastructure with CD conventions. Things I've found better than Jenkins once an organization lets go of over-engineered processes: - TeamCity - Concourse CI - Circle CI - ArgoCD - Tekton For Windows build agents I've met plenty of people successful with Octopus Deploy that have hated Jenkins and your data point is the first I've seen that went back to Jenkins. Perhaps it's worth looking at Team Foundation Server / Azure DevOps?
orlovs 5y ago
Don't abuse CI/CD system as workflow automation tool. There are better and much easier alternatives.
- dfabulich 5y ago
  Such as?
  - oblio 5y ago
    I'd also like to know. Many of them are <<clunkier>> and have smaller communities, less active development, etc. Far from a clear win.
  - otabdeveloper4 5y ago
    (crickets) We don't use Jenkins for CI/CD. It's probably too overkill for those basic tasks. But as the middle ground between crontabs and something like Airflow it fixed all our problems without creating new ones. Are there really any alternatives out there? (And no, we don't need Kubernetes, thanks.)
  - devonkim 5y ago
    I think the question itself begets more questions. For example, I've seen Jenkins jobs replaced with an Apache Airflow DAG or two, but obviously that's not the right call for continuously deploying applications to K8S clusters, which is where I'd use Tekton, Harness, or Argo CD. If we go way back, companies used to churn out Perl and Ruby scripts and eventually they were replaced with anything from Python to Go to Rust. It always goes back to _why_ someone finds deficiencies in what Jenkins jobs provide. Ask 10 people that hate Jenkins and one is likely to find 10 different products / solutions appropriate for them. Most folks I've seen replacing Jenkins for a job runner wind up using Ansible, Salt, or Rundeck if they're not in a giant enterprise (there's also the old HP Operations Orchestration system but that's buried now unfortunately). All of these have their own holy wars brewing and deficiencies, sure, but I personally prefer those warts over adding more maintenance issues by adding in Jenkins. I'd also suggest StackStorm for a more modern, async approach to orchestration workflows. The workflow software ecosystem is built around business domain specialization and it's kind of silly to try to go against Jenkins toe to toe as a company, so this is what we've got sadly. After all, Jetbrains makes TeamCity which is essentially a Better Architected than Jenkins system but it's definitely not very popular either.
clutchdude 5y ago
As someone who runs a large build cluster at a major corp, Jenkins is clunky and outdated. It's a pain to deploy and keep maintained. I'd rather stub my toe than figure out why some event isn't being properly processed in the bowels of Jenkins. However, when coupled with Kubernetes, Jenkins is more powerful than almost any other CI tool. You can orchestrate any iteration of build types or actions while easily providing the underlying resources transparently to your users. Notice that almost every other CI tool mentioned here is either 3rd party hosted or severely limited in some fashion when compared to Jenkins.
- Aeolun 5y ago
  If you are already running in kubernetes, how is any other tool severely hampered if it also runs in kubernetes?
  - clutchdude 5y ago
    It's partially because I'm not as familiar with other tools but they don't expose the ability to interact with an entire pod within the build. Gitlab Action Services for instance is closer to Docker Compose.
deleted 5y ago
[deleted]

munk-a 5y ago

I think Jenkins is a very powerful tool but I would strongly disagree that it's quite valuable in a startup environment. Jenkins has struck me as a philosophical sibling as C++ - it offers a plethora of foot guns to the user that can also be used to accomplish rather good things. I would advise Jenkins to companies with projects that don't require agile responses to user requirements where the functionality desired at the end is very clear from the beginning - since the times I see Jenkins fail the most are when the configuration of jobs on it are being updated and reconfigured rapidly. Jenkins tends to suffer from extremely poor state management and experimenting with things can cause a permanent loss of value to companies if backups are not properly configured - compared with a system like ansible where recipes will grow and be regularly committed to a repository Jenkins doesn't have a nature version control system and as a result strikes me as extremely brittle. I almost wish the UI for jenkins simply didn't exist and it actually was just a whole mess of config files and shell scripts so that it could be locked down to a much better degree.

zug_zug 5y ago
I don't care to unpack most of that. But what I will say is that I recommend automatically backup up jenkins nightly, I accomplished this simply by commiting jenkins_home to a git repo and pushing every night.
- oblio 5y ago
  These days you can also use the Configuration as Code plugin and you can automate the setup, version control it, document it, etc: https://www.jenkins.io/projects/jcasc/ https://www.jenkins.io/projects/jcasc/
- 2rsf 5y ago
  But did you try to restore from backup? I did and was flawless and smooth

aranelsurion 5y ago

Seems to me people hating on Jenkins are not hating on the idea of a central automation service/task runner, they hate on Jenkins itself.

jordanbeiber 5y ago

You could use a tool desgned for it and probably be better of, since upgrading jenkins and plugins, as everyone knows... is not always exactly pain free. Regarding your batch scheduling solution - take hashicorp nomad for a spin: a single go binary and your scheduled job can be declared in 10 lines of yaml. Won’t miss a beat.

TomBombadildoze 5y ago
This isn't a batch scheduling problem. It's a monitoring and alerting problem. OP needs a monitoring and alerting solution. Stop ~~~ engineering ~~~ things and choose the right tool for the job.
- mixologic 5y ago
  "Solution" and "Job" may be overstating things here. If all op needs is monitoring and alerting of a couple of machines, then Datadog/New Relic/Ops Genie/Pager Duty/Server Density/Pingdom/AppDynamics/Loupe/Sysdig/Dynatrace (just to name a few in the crowded space) are all likely overkill and not worth the cost. A large portion of the cost of many of these tools is spent "choosing the right tool for the job" Figuring out what they do, what they do well, where they overlap, where the company that makes the tool is headed, how hard its going to be to swap that tool out for a better one (or cheaper one) - thats a lot of expense in labor and training.
- jordanbeiber 5y ago
  A script scheduled to test something and notify? Hardly ”engineered”. I claim using pretty much any tool other than jenkins is easier to maintain over time (as illustrated in this very article).

codeduck 5y ago

I've been using jenkins for over 10 years. There is no finer system for turning quick CI hacks into entrenched technical debt. Jenkins gives you the freedom to do things without the burden of asking whether you should do those things. Jenkins permits you to munge a whole lot of separate domains into a single, creaky system. There are better tools available for most aspects of CICD and automated testing these days.

dijit 5y ago
> There are better tools available for most aspects of CICD and automated testing these days. It's easy to say this and not cite examples. I rock perforce and windows agents, requirements of the environment, and we struggled very hard to find something that beats Jenkins. One guy even bought an Octopus Deploy license for a year (at the tune of a few thousand dollary-doos) and it did everything worse than our Jenkins setup. I actually dislike Jenkins, I'm open to suggestions here.
- devonkim 5y ago
  Jenkins to me is great for prototyping certain process activities in a very arbitrary set of conventions, but it provides a lesson as to why the Java world moved on from tools like Apache Ant (XML-based Make) to Maven (convention over configuration as a tooling value). While 10 years ago such processes were not really standardized nowadays people are used to conventions like GitOps with well documented processes for how artifacts are built, deployed, and operated with container primitives after we've spent a _lot_ of time now trying to manage Jenkins installations that become incredibly brittle over time and become more of a liability than a force multiplier for the organization. So people are adopting more specialized tooling instead of trying to have Jenkins do everything under the sun from building artifacts to deploying them to deploying infrastructure with CD conventions. Things I've found better than Jenkins once an organization lets go of over-engineered processes: - TeamCity - Concourse CI - Circle CI - ArgoCD - Tekton For Windows build agents I've met plenty of people successful with Octopus Deploy that have hated Jenkins and your data point is the first I've seen that went back to Jenkins. Perhaps it's worth looking at Team Foundation Server / Azure DevOps?

orlovs 5y ago

Don't abuse CI/CD system as workflow automation tool. There are better and much easier alternatives.

dfabulich 5y ago
Such as?
- oblio 5y ago
  I'd also like to know. Many of them are <<clunkier>> and have smaller communities, less active development, etc. Far from a clear win.
- otabdeveloper4 5y ago
  (crickets) We don't use Jenkins for CI/CD. It's probably too overkill for those basic tasks. But as the middle ground between crontabs and something like Airflow it fixed all our problems without creating new ones. Are there really any alternatives out there? (And no, we don't need Kubernetes, thanks.)
- devonkim 5y ago
  I think the question itself begets more questions. For example, I've seen Jenkins jobs replaced with an Apache Airflow DAG or two, but obviously that's not the right call for continuously deploying applications to K8S clusters, which is where I'd use Tekton, Harness, or Argo CD. If we go way back, companies used to churn out Perl and Ruby scripts and eventually they were replaced with anything from Python to Go to Rust. It always goes back to _why_ someone finds deficiencies in what Jenkins jobs provide. Ask 10 people that hate Jenkins and one is likely to find 10 different products / solutions appropriate for them. Most folks I've seen replacing Jenkins for a job runner wind up using Ansible, Salt, or Rundeck if they're not in a giant enterprise (there's also the old HP Operations Orchestration system but that's buried now unfortunately). All of these have their own holy wars brewing and deficiencies, sure, but I personally prefer those warts over adding more maintenance issues by adding in Jenkins. I'd also suggest StackStorm for a more modern, async approach to orchestration workflows. The workflow software ecosystem is built around business domain specialization and it's kind of silly to try to go against Jenkins toe to toe as a company, so this is what we've got sadly. After all, Jetbrains makes TeamCity which is essentially a Better Architected than Jenkins system but it's definitely not very popular either.

clutchdude 5y ago

As someone who runs a large build cluster at a major corp, Jenkins is clunky and outdated. It's a pain to deploy and keep maintained. I'd rather stub my toe than figure out why some event isn't being properly processed in the bowels of Jenkins. However, when coupled with Kubernetes, Jenkins is more powerful than almost any other CI tool. You can orchestrate any iteration of build types or actions while easily providing the underlying resources transparently to your users. Notice that almost every other CI tool mentioned here is either 3rd party hosted or severely limited in some fashion when compared to Jenkins.

Aeolun 5y ago
If you are already running in kubernetes, how is any other tool severely hampered if it also runs in kubernetes?
- clutchdude 5y ago
  It's partially because I'm not as familiar with other tools but they don't expose the ability to interact with an entire pod within the build. Gitlab Action Services for instance is closer to Docker Compose.

deleted 5y ago

[deleted]