7 ms·

If you're using yaml and templates then you've already lost. The only tool in this game that is not braindead is chef. Sometimes you need imperative things and

by _qc3o 11y ago

If you're using yaml and templates then you've already lost. The only tool in this game that is not braindead is chef. Sometimes you need imperative things and conditional logic with iteration. If you don't have a real programming language then the contortions you have to go through gets really old really fast. As for the deployment patterns. If you're in the cloud then you should be baking AMIs (or equivalent in your cloud provider) and shipping your configuration the same way you ship your application code, as native packages like .deb or .rpm. If you jumped on the docker bandwagon then your hosts are basically there to look pretty and host the containers which means you have some other way of getting configuration to your servers, i.e. etcd, consul, etc. so the problems brought up in this post don't exist in that setting. You are also probably using some kind of container orchestration system like kubernetes so again the problem of orchestration and deployment is offloaded to some other system. The only problem you have in that setting is doing a rolling deploy of containers and halting when things go wrong. I think the only place any of these tools make sense now is some private on-premise cloud. Ever other place has already moved on.

yo-code-sucks 11y ago
Yeah because we need more abstraction from the simple things. Fuck everything Chef.
smw 11y ago
Yes! Please, for the love of all that is holy, please quit writing tools that make me write code in something that isn't a general purpose programming language! Didn't we learn from Ant and xml? I really want to see a clojure/clojurescript based config management system -- it would be so pleasant to write EDN/sexps for basic config, and yet have it be a real language when you need to do something hard. Edit: Forgot to mention, pallet [1] is something like that, but unfortunately it appears to be mostly dead. [1]: http://palletops.com/ http://palletops.com/
- dm3 11y ago
  We're using pallet. Even though it's not trivial to get started with and the docs are quite sparse, it provides a solid foundation for scripting the configuration management tasks. I much prefer it to Ansible/Chef/Puppet. However, as you're exposed to the full programming language, there are much less constraints to what you can do. You'll need a layer of rules (a framework) of your own if you don't want to end up in a mess. Activity of the core project seems to be low, but that's true for many other mature Clojure projects. You can still use them successfully.
- vacri 11y ago
  The only tool I've used that used native language was Buildbot, from which I still have the scars. Not because of the native language, but from the versioned documentation that didn't match behaviour of the given version, and the complex workflow that had zero examples because "everyone's setup is different" - so you had to figure out their words for terms, and build your config from scratch. It was an entirely unpleasant experience.
- crdoconnor 11y ago
  >Yes! Please, for the love of all that is holy, please quit writing tools that make me write code in something that isn't a general purpose programming language! Didn't we learn from Ant and xml? Yes, we learned to use less powerful languages where it was appropriate because they're more readable and less susceptible to technical debt. This principle, in other words : https://en.wikipedia.org/wiki/Rule_of_least_power https://en.wikipedia.org/wiki/Rule_of_least_power Ant XML was as powerful as Java - it was turing complete and terribly designed to boot. That was its primary failing. Likewise, using turing complete PHP to generate HTML was never as clean as using a less powerful templating language (like jinja2) to generate the HTML. Separation of concerns with a language barrier is a good thing. If all of this means nothing to you, you've probably created some huge code messes in your time.
  - Singletoned 11y ago
    > Likewise, using turing complete PHP to generate HTML was never as clean as using a less powerful templating language (like jinja2) to generate the HTML. I strongly agree with your point, but Jinja2 is Turing Complete as well (it's still preferable to PHP though).
    - crdoconnor 11y ago
      I don't really know enough computer science to validate this idea, but I can sense that there are different levels of "power" among turing complete languages (and also among non-turing complete languages). And Jinja2 < Python/PHP, despite all three being turing complete. Metaprogramming / C++ style templating, for instance, goes above and beyond the power provided by regular turing complete programming constructs, and while that means that you can do cool stuff with them you couldn't easily do otherwise, they're a massive headache to reason about, debug, and keep free from technical debt. Similarly, when you take blocks of code and "lower the power" to make it declarative instead of imperative (e.g. using list comprehensions instead of for loops) it almost inevitably ends up cleaner.
  - deleted 11y ago
    [deleted]
- alanning 11y ago
  We also use pallet. Harder to grok in the beginning since you have to learn the lib but it's pretty amazing what you can do with a full language. If anyone is frustrated by the speed of jclouds (which pallet uses internally), I recommend trying pallet's Amazon-specific driver; it's much faster.
viraptor 11y ago
There may be a scale of braindeadness, but chef still scores pretty high on it. For example, chef server cannot tell a difference between a new node and an update of node state. (and that's enforced by the API design, not specific implementation) Just think of all the weird things that can happen this way in a very dynamic environment. (example: if you made some mistake while provisioning new hosts, you'll have one node data overriding another, without any notification)
- vegardx 11y ago
  How do you manage that exactly? Using the same client certificates? Client names are unique, so you can't have duplicate entries of the supposed same node.
  - viraptor 11y ago
    When cloning machines for example you can race to replace the credentials. Also when deleting nodes, you need to make sure you first delete the client, then node - otherwise the node can get silently recreated and mess up your discovery. Also when... (quite a few possibilities).
    - vegardx 11y ago
      Ah, yes. It's somewhat annoying that there isn't a clean way to remove both client and node in one go. But when it comes to cloning the images should be prepped before cloning. Or just don't clone.
nkuttler 11y ago
> The only tool in this game that is not braindead is chef. Yeah, right... https://docs.saltstack.com/en/latest/ref/renderers/all/ https://docs.saltstack.com/en/latest/ref/renderers/all/
- dkarapetyan 11y ago
  So not only do I get to write yaml or some other template nonsense but I get to choose the dialect as well. Yes, much better.
dmourati 11y ago
I'm pro-baking AMIs. I do it every day and have automated and wrapped it up with Jenkins/Packer etc. I don't get your point though. Something, ansible, puppet, something has to put all the config in place. I've done the whole rpm for everything approach before too. It didn't end that well to be honest.
movedx 11y ago
It's clear you're a developer and not a systems engineer.
- dkarapetyan 11y ago
  I like to think I'm a generic problem solver that through accidents of history has ended up using software instead of pencil and paper to solve problems. In another life pencil and paper was adequate. Systems, programming, administration is all the same to me. I don't discriminate. I also like to use actual programming languages instead of their yaml based bastardizations.
  - movedx 11y ago
    "I also like to use actual programming languages instead of their yaml based bastardizations." I think the problem here is confusion, and I see it a lot in the Ansible community. What you're doing, and I can completely understand why, is taking Ansible as being a programming language for defining state. That isn't what Ansible is designed to be. That YAML you speak of is designed to allow you to define state in a format that is not only human readable, but also machine readable. All that YAML is meant to do is say: x=y. it's not trying to be a programming language or a scripting interface, it's simply a means of setting the state you want. This is a common pitfall a lot of people fall into when they approach Ansible, and I'm always seeing questions on IRC like: "How do I code Ansible to pull down a file, read it, process it, and after doing some logic, do X?" The answer is: you're using Ansible wrong. Use it as a state management system and only a state management system, and you won't go wrong. Use a scripting or programming language, write a script/program, and run that on the remote machine if you need highly complex logic to determine state.
- edcastro 11y ago
  That. Indeed there's a whole different mindset for people deploying and orchestrathing servers based on their 'previous' background. On the previous iteration of CM, I noticed that people that had a prior background on development usually chose Chef, while the SysAdms chose Puppet.
bovermyer 11y ago
This seems to be a common problem among us engineers: that we are certain we know the Correct Tool, and everyone else is either stupid, ignorant, or misled. Or all three. Interestingly, I've found that a focus on solving the problem at hand (say, installing a package on a hundred systems quickly and predictably) rather than a focus on using the Absolute Best Tool is much more likely to lead to a pleasant work experience for everyone involved. That, and "the Best" is frequently subjective and, even when accurate, tends to get replaced in a few weeks by something else.
- dkarapetyan 11y ago
  I'm ok with limitations when the limitations actually buy me something. I've used all the tools in the space and they all manage to miss the mark. If I'm giving up programmability then I'd better get something in exchange. If that something is templates that I can't reason about if the right context is not available then that's not solving any problem. First and foremost whatever I use I must be able to reason about and debug. Strings concatenated together with other strings based on some weird rules is anything but reasonable and debuggable.
wtbob 11y ago
> If you're using yaml and templates then you've already lost. Probably true. > The only tool in this game that is not braindead is chef. What's wrong with s-expressions? Code and data at the same time.
- davexunit 11y ago
  >What's wrong with s-expressions? Code and data at the same time. Sure, but you need something that will actually evaluate that sexp. May I suggest GNU Guix to fill that role? http://www.gnu.org/software/guix/ http://www.gnu.org/software/guix/
  - wtbob 11y ago
    I really like Guix's ideas, but I think that project made a very unwise choice when it picked Scheme rather than Common Lisp.

by _qc3o 11y ago

yo-code-sucks 11y ago

Yeah because we need more abstraction from the simple things. Fuck everything Chef.

smw 11y ago

Yes! Please, for the love of all that is holy, please quit writing tools that make me write code in something that isn't a general purpose programming language! Didn't we learn from Ant and xml? I really want to see a clojure/clojurescript based config management system -- it would be so pleasant to write EDN/sexps for basic config, and yet have it be a real language when you need to do something hard. Edit: Forgot to mention, pallet [1] is something like that, but unfortunately it appears to be mostly dead. [1]: http://palletops.com/ http://palletops.com/

dm3 11y ago
We're using pallet. Even though it's not trivial to get started with and the docs are quite sparse, it provides a solid foundation for scripting the configuration management tasks. I much prefer it to Ansible/Chef/Puppet. However, as you're exposed to the full programming language, there are much less constraints to what you can do. You'll need a layer of rules (a framework) of your own if you don't want to end up in a mess. Activity of the core project seems to be low, but that's true for many other mature Clojure projects. You can still use them successfully.
vacri 11y ago
The only tool I've used that used native language was Buildbot, from which I still have the scars. Not because of the native language, but from the versioned documentation that didn't match behaviour of the given version, and the complex workflow that had zero examples because "everyone's setup is different" - so you had to figure out their words for terms, and build your config from scratch. It was an entirely unpleasant experience.
crdoconnor 11y ago
>Yes! Please, for the love of all that is holy, please quit writing tools that make me write code in something that isn't a general purpose programming language! Didn't we learn from Ant and xml? Yes, we learned to use less powerful languages where it was appropriate because they're more readable and less susceptible to technical debt. This principle, in other words : https://en.wikipedia.org/wiki/Rule_of_least_power https://en.wikipedia.org/wiki/Rule_of_least_power Ant XML was as powerful as Java - it was turing complete and terribly designed to boot. That was its primary failing. Likewise, using turing complete PHP to generate HTML was never as clean as using a less powerful templating language (like jinja2) to generate the HTML. Separation of concerns with a language barrier is a good thing. If all of this means nothing to you, you've probably created some huge code messes in your time.
- Singletoned 11y ago
  > Likewise, using turing complete PHP to generate HTML was never as clean as using a less powerful templating language (like jinja2) to generate the HTML. I strongly agree with your point, but Jinja2 is Turing Complete as well (it's still preferable to PHP though).
  - crdoconnor 11y ago
    I don't really know enough computer science to validate this idea, but I can sense that there are different levels of "power" among turing complete languages (and also among non-turing complete languages). And Jinja2 < Python/PHP, despite all three being turing complete. Metaprogramming / C++ style templating, for instance, goes above and beyond the power provided by regular turing complete programming constructs, and while that means that you can do cool stuff with them you couldn't easily do otherwise, they're a massive headache to reason about, debug, and keep free from technical debt. Similarly, when you take blocks of code and "lower the power" to make it declarative instead of imperative (e.g. using list comprehensions instead of for loops) it almost inevitably ends up cleaner.
- deleted 11y ago
  [deleted]
alanning 11y ago
We also use pallet. Harder to grok in the beginning since you have to learn the lib but it's pretty amazing what you can do with a full language. If anyone is frustrated by the speed of jclouds (which pallet uses internally), I recommend trying pallet's Amazon-specific driver; it's much faster.

viraptor 11y ago

There may be a scale of braindeadness, but chef still scores pretty high on it. For example, chef server cannot tell a difference between a new node and an update of node state. (and that's enforced by the API design, not specific implementation) Just think of all the weird things that can happen this way in a very dynamic environment. (example: if you made some mistake while provisioning new hosts, you'll have one node data overriding another, without any notification)

vegardx 11y ago
How do you manage that exactly? Using the same client certificates? Client names are unique, so you can't have duplicate entries of the supposed same node.
- viraptor 11y ago
  When cloning machines for example you can race to replace the credentials. Also when deleting nodes, you need to make sure you first delete the client, then node - otherwise the node can get silently recreated and mess up your discovery. Also when... (quite a few possibilities).
  - vegardx 11y ago
    Ah, yes. It's somewhat annoying that there isn't a clean way to remove both client and node in one go. But when it comes to cloning the images should be prepped before cloning. Or just don't clone.

nkuttler 11y ago

> The only tool in this game that is not braindead is chef. Yeah, right... https://docs.saltstack.com/en/latest/ref/renderers/all/ https://docs.saltstack.com/en/latest/ref/renderers/all/

dkarapetyan 11y ago
So not only do I get to write yaml or some other template nonsense but I get to choose the dialect as well. Yes, much better.

dmourati 11y ago

I'm pro-baking AMIs. I do it every day and have automated and wrapped it up with Jenkins/Packer etc. I don't get your point though. Something, ansible, puppet, something has to put all the config in place. I've done the whole rpm for everything approach before too. It didn't end that well to be honest.

movedx 11y ago

It's clear you're a developer and not a systems engineer.

dkarapetyan 11y ago
I like to think I'm a generic problem solver that through accidents of history has ended up using software instead of pencil and paper to solve problems. In another life pencil and paper was adequate. Systems, programming, administration is all the same to me. I don't discriminate. I also like to use actual programming languages instead of their yaml based bastardizations.
- movedx 11y ago
  "I also like to use actual programming languages instead of their yaml based bastardizations." I think the problem here is confusion, and I see it a lot in the Ansible community. What you're doing, and I can completely understand why, is taking Ansible as being a programming language for defining state. That isn't what Ansible is designed to be. That YAML you speak of is designed to allow you to define state in a format that is not only human readable, but also machine readable. All that YAML is meant to do is say: x=y. it's not trying to be a programming language or a scripting interface, it's simply a means of setting the state you want. This is a common pitfall a lot of people fall into when they approach Ansible, and I'm always seeing questions on IRC like: "How do I code Ansible to pull down a file, read it, process it, and after doing some logic, do X?" The answer is: you're using Ansible wrong. Use it as a state management system and only a state management system, and you won't go wrong. Use a scripting or programming language, write a script/program, and run that on the remote machine if you need highly complex logic to determine state.
edcastro 11y ago
That. Indeed there's a whole different mindset for people deploying and orchestrathing servers based on their 'previous' background. On the previous iteration of CM, I noticed that people that had a prior background on development usually chose Chef, while the SysAdms chose Puppet.

bovermyer 11y ago

This seems to be a common problem among us engineers: that we are certain we know the Correct Tool, and everyone else is either stupid, ignorant, or misled. Or all three. Interestingly, I've found that a focus on solving the problem at hand (say, installing a package on a hundred systems quickly and predictably) rather than a focus on using the Absolute Best Tool is much more likely to lead to a pleasant work experience for everyone involved. That, and "the Best" is frequently subjective and, even when accurate, tends to get replaced in a few weeks by something else.

dkarapetyan 11y ago
I'm ok with limitations when the limitations actually buy me something. I've used all the tools in the space and they all manage to miss the mark. If I'm giving up programmability then I'd better get something in exchange. If that something is templates that I can't reason about if the right context is not available then that's not solving any problem. First and foremost whatever I use I must be able to reason about and debug. Strings concatenated together with other strings based on some weird rules is anything but reasonable and debuggable.

wtbob 11y ago

> If you're using yaml and templates then you've already lost. Probably true. > The only tool in this game that is not braindead is chef. What's wrong with s-expressions? Code and data at the same time.

davexunit 11y ago
>What's wrong with s-expressions? Code and data at the same time. Sure, but you need something that will actually evaluate that sexp. May I suggest GNU Guix to fill that role? http://www.gnu.org/software/guix/ http://www.gnu.org/software/guix/
- wtbob 11y ago
  I really like Guix's ideas, but I think that project made a very unwise choice when it picked Scheme rather than Common Lisp.