6 ms·

I'm skeptical, because there is already a configuration management language that is strongly influenced by Prolog and logic programming - Puppet - and the resul

I'm skeptical, because there is already a configuration management language that is strongly influenced by Prolog and logic programming - Puppet - and the results are not that great. The secret of configuration management is that ordering NEEDS to be a first class feature and ordered manifests should be the default, not the option. Puppet suffers hugely from the influence from the fact that ordering is unpredictable and that you have to use arrows or requires to add order. People accidentally forget to add ordering, the resulting code works most of the time, and then one day you start running into an issue because some implicit dependency either didn't happen, or happened in the wrong order. In as much as all logic programming involves not explicitly controlling flow, I think they're inappropriate for configuration management. Sadly, I'm still stuck with using Puppet because of the organisation I work in. Having worked with Prolog in the past, I would also suggest that an important feature of a configuration management language is that the syntax make it hard to make typos and other minor errors because the feedback loop of deploying your code against a local vagrant machine is already long enough. I vividly remember the annoyance of having missed the full stop to end a block. I'm not desperate to repeat that experience.

gwillem 13y ago
That's the main reason we've switched over from Puppet to Ansible. It has less abstraction layers, but at least its execution order is predictable. The easy learning curve for new colleagues and the compact syntax are also nice benefits.
- lobster_johnson 13y ago
  That's what I don't understand about Ansible. It's basically -- literally, actually -- Puppet reinvented with a new syntax based on YAML. Puppet works so well, Ansible seems like a typical "not invented here" tragedy that frequently occurs in open source. Rather than coalesce around something that works and has years of experience and knowledge and bug fixing sunk into it, people create something from scratch. To be specific, this is Ansible: - name: Memcached template: src=$repository_basedir/memcached/templates/memcached.conf.j2 dest=/etc/memcached.conf owner=root group=root mode=0644 notify: - memcached-restart The same in Puppet: file { '/etc/memcached.conf': content => template('memcached/memcached.conf'), owner => root, group => root, mode => '0644', notify => Service['memcached']; } It's literally the same data expressed here. Puppet has very little abstraction. But what you do get, which Ansible has not really understood, I think, is a data model. The above results in the graph getting an object File['/etc/memcached.conf'] which can be referred to. I can do this: service { 'memcached': require => File['/etc/memcached.conf']; } This in turn creates the graph node Service['memcached']. It takes part in a larger graph that looks like this: Stage[main]/Node[myserver]/Service['memcached'] In fact, manifests are all compiled into a large graph like this, with interdependencies between the nodes. The logic-based, Prolog-type language in the OP's link is precisely what Puppet builds underneath. Likewise, the graph model is extensible. I can invent a new object like so: define public_key() { file { "/home/$name/.ssh/authorized_keys": content => "$key"; } } public_key { 'someuser': key => 'some public key'; } account { 'someuser': require => PublicKey['someuser']; } Here, the define public_key() actually defines a new type of object that can be instantiated in scripts. When one is instantiated, other objects can be depend on it. This allows me to encapsulate stuff in my scripts; rather than having other declarations depend on the internal details of how a public key is stored (ie., in a home directory), they depend on my definition. Ansible may look simpler on the surface, but I think it's all surface. When you have learned to use Puppet, it becomes simple. Ansible looks to me like the product of a person who looked at Puppet, said something like "ugh Ruby, ew syntax" and made his own thing, without looking deeper. Simialrly, Ansible appeals to people who look at Puppet with the same reaction, again without looking deeper. The irony is that they are buying into something that is arguably poorer. That said, I would argue that part of the problem is Puppet's documentation, which surely must drive people to "poor man's Puppet" solutions. (The cynic in me wonders if Puppet's poor documentation is a deliberate ploy on part of Puppet Labs to boost their consulting business.)
  - rdtsc 13y ago
    > Puppet works so well, Ansible seems like a typical "not invented here" tragedy that frequently occurs in open source. That is a bit kind of what I get from Ansible. Ok at first the best thing it had going for it was 'it works over plain SSH and uses YAML'. Basically for those who did everything via SSH and shell scripting. This was an upgrade. Now once they get into complicated playbooks and transitive dependencies, 0mq daemons, yes it looks like it is just another Puppet but newer (some kids just like newer things because they want to feel like they are at the cutting edge of technology). There is also Salt. That had the "look really fast and responsive configuration" because it has the 0mq based distribution mechanism. But then Ansible added that too as a feature. Salt looked at and said "ok fine, we'll add SSH only option too". So now they are both basically solving a similar problem along with the older tools (Puppet, Chef etc)
    - calpaterson 13y ago
      Haven't used Salt much, as I understand it they do take Puppets model. Even if you want that (and I don't) there are still a lot of ways you could do a better job, so I'm somewhat open to Salt. However, Ansible is significantly different from Puppet. For starters it is push-based rather than pull-based as a lot of Puppet setups are. The other important difference is that Ansible has top-to-bottom ordering. That's what first attracted me.
      - lamontcg 13y ago
        Push based SSH models fail pretty badly when you starting talking hundreds or thousands of servers under management. Those models have existed for a log time in tools like rdist, and sitescope tried to sell 'agentless' monitoring which required SSH in order to monitor servers. I've seen deployments of sitescope like that hit scalability walls at a few hundred servers as the ssh client processes on the sitescope server were chewing all the CPU doing SSH session negotiation. And the point here is that 'agentless' is really a lie. You're not agentless, you're using sshd as your agent. And when you dig into that protocol its expensive and not designed for RPC, its designed for interactive logins, and its chatty and chews up resources. It also isn't terribly reliable. It seems reliable when you're using it on your workstation and logging into a few dozen servers a day. You don't remember that time a few months ago when it had a 'derp' and you had to login again. When you're hitting 30,000 servers multiple times a day, you start to see the unreliability of the underlying protocol. If Ansible is going to survive you're going to see some kind of 'persistent fireball mode' where you ssh into the box and leave the zeromq daemon running and it starts looking a whole lot more like push. And really you want nodes registering back to your server. It makes discovery so much easier since you just configure your chef/puppet/cfengine/whatever server in whatever bootstrapping scheme you're using (kickstart, custom amis, whatever) and then whenever a server comes up, it registers and starts working. The push method where you have to edit a file on a server which is used to push code just leads to crufty servers getting untracked. The idea isn't really new, its been around for decades, and its always the first baby steps on the road to doing config management.
NuclearTide 13y ago
I second everything said here. In my internship last summer, I arbitrarily chose Puppet to automate the configuration of the company's developer environments. Puppet's unpredictable ordering should be a write-and-forget benefit, but in reality it's very difficult to reason about. In the end, my script imposed a top-to-bottom ordering with requires. Also, developing configuration scripts against Vagrant is a huge pain. While `puppet parser validate` checks for syntax errors, most of the logical, dependency-based errors reveal themselves after Vagrant takes a good minute to fire up. Anyway, hopefully my Puppet-Vagrant scripts aren't causing any maintenance nightmares...
- robbyt 13y ago
  If reasoning about explicit order of resources is too much for you, just enable the new resource ordering feature: http://docs.puppetlabs.com/references/latest/configuration.html#ordering http://docs.puppetlabs.com/references/latest/configuration.h... Then assign classes like: class{'ntp':}-> class{'ssh':}-> class{'other_stuff':} problem solved?
  - calpaterson 13y ago
    Sure, except that once you're doing stuff like that, you might as well switch to a manifest-ordered tool like Ansible. Ordering isn't the only problem with Puppet, just the most serious.
    - robbyt 13y ago
      What are the other problems with Puppet?
- danieldk 13y ago
  Puppet's unpredictable ordering should be a write-and-forget benefit, but in reality it's very difficult to reason about. But this is Prolog and ordering in Prolog is deterministic. Clause goals are proven in order. When there are multiple clauses with the same predicate, they are executed in order. If this weren't the case extra-logical facilities such as cuts wouldn't work in Prolog
mitchellh 13y ago
I have to respectfully disagree. I try not to take sides in these issues because the software I make all works well and co-exists with Puppet, Chef, etc. But I feel I need to voice my opinion here. I _love_ the explicit ordering of Puppet. It is one of my favorite features over other options. I'll explain why I think it is a good thing. For credentials to add some sort of credibility to this post: I worked for 4 years as an ops person managing not small (but not big) infrastructures for multiple companies ranging from 10 nodes up to 200 nodes. Some organizations used Chef, some used Puppet, and I even used Salt once. I created and maintain Vagrant and Packer, both of which integrate deeply with all major configuration management systems, so I have to know how they work decently well to be able to maintain that. Let's use Chef for the remainder of this post, but please don't take this post as saying "Chef is bad. Puppet is good." That is false. I'm just using Chef as an example where ordering is implicit (based on the order resources appear in a recipe and run list). The explicit ordering of Puppet makes it very clear exactly what a resource depends on. When training junior ops, I was frequently asked "so I use requires/notify/etc. to tell Puppet what order to do things?" And I would say NO! That is the _result_, but the purpose of requires is to list what the resource has as a logical _dependency_. For example, a common problem with junior puppet devs is to say "Oh, I'm making resource C which depends on resource B and A, but since B depends on A, I'll just put 'B' in the requires list." That is _wrong_. Logically, C depends on B and A, so put both there! Don't think about ordering, think about requirements/dependencies. Given this, Puppet is so great. You can look at any resource and say "oh hey, this depends on ONLY these list of things." Sure, the dependencies may have their own dependencies, but the resource in question doesn't care about that. Example: this file requires that this service is installed. That service might require, say, RubyGems, but the file doesn't care! Its not an immediate dependency of the file. With Chef, on the other hand, I would see "okay, C happens after B and A, but does C _really_ need to happen after A?" I don't know. I can't answer that. It requires much deeper domain knowledge of that infrastructure to answer that question. This leads to more of a nightmare [for me] months down the road, when I don't quite remember if something needs to happen after something else. With Puppet, as a style rule, I require all ops engineers list _all_ requirements (exhaustive require clauses) whenever they're working. Then, months down the road, you can move around anything you want. You know if you broke something because Puppet will tell you (before it runs, during the 'compilation' phase). Personally, I prefer this sort of explicitness. I understand that some don't like this, but I like it. And people can like it. I don't believe either way is STRICTLY better/worse. I felt my opinion needed to be shared.
lars512 13y ago
Even in "non-deterministic" style prolog, the execution order is still fixed, and dependencies are traversed in the order you wrote them. There will be no surprises. The missing full-stop is no different to a missing semi-colon, comma, or backslash, issues we still have in many modern languages.
- calpaterson 13y ago
  > The missing full-stop is no different to a missing semi-colon Yeah, it's an annoyance. The difference is that the feedback loop on operations codebases is longer, so they're more annoying.
char_pointer 13y ago
For Puppet you can use a validation script and a git commit hook to shorten the feedback loop on syntax errors: http://spoop.net/devops/2013/11/12/validate-puppet.html http://spoop.net/devops/2013/11/12/validate-puppet.html

gwillem 13y ago

That's the main reason we've switched over from Puppet to Ansible. It has less abstraction layers, but at least its execution order is predictable. The easy learning curve for new colleagues and the compact syntax are also nice benefits.

lobster_johnson 13y ago
That's what I don't understand about Ansible. It's basically -- literally, actually -- Puppet reinvented with a new syntax based on YAML. Puppet works so well, Ansible seems like a typical "not invented here" tragedy that frequently occurs in open source. Rather than coalesce around something that works and has years of experience and knowledge and bug fixing sunk into it, people create something from scratch. To be specific, this is Ansible: - name: Memcached template: src=$repository_basedir/memcached/templates/memcached.conf.j2 dest=/etc/memcached.conf owner=root group=root mode=0644 notify: - memcached-restart The same in Puppet: file { '/etc/memcached.conf': content => template('memcached/memcached.conf'), owner => root, group => root, mode => '0644', notify => Service['memcached']; } It's literally the same data expressed here. Puppet has very little abstraction. But what you do get, which Ansible has not really understood, I think, is a data model. The above results in the graph getting an object File['/etc/memcached.conf'] which can be referred to. I can do this: service { 'memcached': require => File['/etc/memcached.conf']; } This in turn creates the graph node Service['memcached']. It takes part in a larger graph that looks like this: Stage[main]/Node[myserver]/Service['memcached'] In fact, manifests are all compiled into a large graph like this, with interdependencies between the nodes. The logic-based, Prolog-type language in the OP's link is precisely what Puppet builds underneath. Likewise, the graph model is extensible. I can invent a new object like so: define public_key() { file { "/home/$name/.ssh/authorized_keys": content => "$key"; } } public_key { 'someuser': key => 'some public key'; } account { 'someuser': require => PublicKey['someuser']; } Here, the define public_key() actually defines a new type of object that can be instantiated in scripts. When one is instantiated, other objects can be depend on it. This allows me to encapsulate stuff in my scripts; rather than having other declarations depend on the internal details of how a public key is stored (ie., in a home directory), they depend on my definition. Ansible may look simpler on the surface, but I think it's all surface. When you have learned to use Puppet, it becomes simple. Ansible looks to me like the product of a person who looked at Puppet, said something like "ugh Ruby, ew syntax" and made his own thing, without looking deeper. Simialrly, Ansible appeals to people who look at Puppet with the same reaction, again without looking deeper. The irony is that they are buying into something that is arguably poorer. That said, I would argue that part of the problem is Puppet's documentation, which surely must drive people to "poor man's Puppet" solutions. (The cynic in me wonders if Puppet's poor documentation is a deliberate ploy on part of Puppet Labs to boost their consulting business.)
- rdtsc 13y ago
  > Puppet works so well, Ansible seems like a typical "not invented here" tragedy that frequently occurs in open source. That is a bit kind of what I get from Ansible. Ok at first the best thing it had going for it was 'it works over plain SSH and uses YAML'. Basically for those who did everything via SSH and shell scripting. This was an upgrade. Now once they get into complicated playbooks and transitive dependencies, 0mq daemons, yes it looks like it is just another Puppet but newer (some kids just like newer things because they want to feel like they are at the cutting edge of technology). There is also Salt. That had the "look really fast and responsive configuration" because it has the 0mq based distribution mechanism. But then Ansible added that too as a feature. Salt looked at and said "ok fine, we'll add SSH only option too". So now they are both basically solving a similar problem along with the older tools (Puppet, Chef etc)
  - calpaterson 13y ago
    Haven't used Salt much, as I understand it they do take Puppets model. Even if you want that (and I don't) there are still a lot of ways you could do a better job, so I'm somewhat open to Salt. However, Ansible is significantly different from Puppet. For starters it is push-based rather than pull-based as a lot of Puppet setups are. The other important difference is that Ansible has top-to-bottom ordering. That's what first attracted me.
    - lamontcg 13y ago
      Push based SSH models fail pretty badly when you starting talking hundreds or thousands of servers under management. Those models have existed for a log time in tools like rdist, and sitescope tried to sell 'agentless' monitoring which required SSH in order to monitor servers. I've seen deployments of sitescope like that hit scalability walls at a few hundred servers as the ssh client processes on the sitescope server were chewing all the CPU doing SSH session negotiation. And the point here is that 'agentless' is really a lie. You're not agentless, you're using sshd as your agent. And when you dig into that protocol its expensive and not designed for RPC, its designed for interactive logins, and its chatty and chews up resources. It also isn't terribly reliable. It seems reliable when you're using it on your workstation and logging into a few dozen servers a day. You don't remember that time a few months ago when it had a 'derp' and you had to login again. When you're hitting 30,000 servers multiple times a day, you start to see the unreliability of the underlying protocol. If Ansible is going to survive you're going to see some kind of 'persistent fireball mode' where you ssh into the box and leave the zeromq daemon running and it starts looking a whole lot more like push. And really you want nodes registering back to your server. It makes discovery so much easier since you just configure your chef/puppet/cfengine/whatever server in whatever bootstrapping scheme you're using (kickstart, custom amis, whatever) and then whenever a server comes up, it registers and starts working. The push method where you have to edit a file on a server which is used to push code just leads to crufty servers getting untracked. The idea isn't really new, its been around for decades, and its always the first baby steps on the road to doing config management.

NuclearTide 13y ago

I second everything said here. In my internship last summer, I arbitrarily chose Puppet to automate the configuration of the company's developer environments. Puppet's unpredictable ordering should be a write-and-forget benefit, but in reality it's very difficult to reason about. In the end, my script imposed a top-to-bottom ordering with requires. Also, developing configuration scripts against Vagrant is a huge pain. While `puppet parser validate` checks for syntax errors, most of the logical, dependency-based errors reveal themselves after Vagrant takes a good minute to fire up. Anyway, hopefully my Puppet-Vagrant scripts aren't causing any maintenance nightmares...

robbyt 13y ago
If reasoning about explicit order of resources is too much for you, just enable the new resource ordering feature: http://docs.puppetlabs.com/references/latest/configuration.html#ordering http://docs.puppetlabs.com/references/latest/configuration.h... Then assign classes like: class{'ntp':}-> class{'ssh':}-> class{'other_stuff':} problem solved?
- calpaterson 13y ago
  Sure, except that once you're doing stuff like that, you might as well switch to a manifest-ordered tool like Ansible. Ordering isn't the only problem with Puppet, just the most serious.
  - robbyt 13y ago
    What are the other problems with Puppet?
danieldk 13y ago
Puppet's unpredictable ordering should be a write-and-forget benefit, but in reality it's very difficult to reason about. But this is Prolog and ordering in Prolog is deterministic. Clause goals are proven in order. When there are multiple clauses with the same predicate, they are executed in order. If this weren't the case extra-logical facilities such as cuts wouldn't work in Prolog

mitchellh 13y ago

I have to respectfully disagree. I try not to take sides in these issues because the software I make all works well and co-exists with Puppet, Chef, etc. But I feel I need to voice my opinion here. I _love_ the explicit ordering of Puppet. It is one of my favorite features over other options. I'll explain why I think it is a good thing. For credentials to add some sort of credibility to this post: I worked for 4 years as an ops person managing not small (but not big) infrastructures for multiple companies ranging from 10 nodes up to 200 nodes. Some organizations used Chef, some used Puppet, and I even used Salt once. I created and maintain Vagrant and Packer, both of which integrate deeply with all major configuration management systems, so I have to know how they work decently well to be able to maintain that. Let's use Chef for the remainder of this post, but please don't take this post as saying "Chef is bad. Puppet is good." That is false. I'm just using Chef as an example where ordering is implicit (based on the order resources appear in a recipe and run list). The explicit ordering of Puppet makes it very clear exactly what a resource depends on. When training junior ops, I was frequently asked "so I use requires/notify/etc. to tell Puppet what order to do things?" And I would say NO! That is the _result_, but the purpose of requires is to list what the resource has as a logical _dependency_. For example, a common problem with junior puppet devs is to say "Oh, I'm making resource C which depends on resource B and A, but since B depends on A, I'll just put 'B' in the requires list." That is _wrong_. Logically, C depends on B and A, so put both there! Don't think about ordering, think about requirements/dependencies. Given this, Puppet is so great. You can look at any resource and say "oh hey, this depends on ONLY these list of things." Sure, the dependencies may have their own dependencies, but the resource in question doesn't care about that. Example: this file requires that this service is installed. That service might require, say, RubyGems, but the file doesn't care! Its not an immediate dependency of the file. With Chef, on the other hand, I would see "okay, C happens after B and A, but does C _really_ need to happen after A?" I don't know. I can't answer that. It requires much deeper domain knowledge of that infrastructure to answer that question. This leads to more of a nightmare [for me] months down the road, when I don't quite remember if something needs to happen after something else. With Puppet, as a style rule, I require all ops engineers list _all_ requirements (exhaustive require clauses) whenever they're working. Then, months down the road, you can move around anything you want. You know if you broke something because Puppet will tell you (before it runs, during the 'compilation' phase). Personally, I prefer this sort of explicitness. I understand that some don't like this, but I like it. And people can like it. I don't believe either way is STRICTLY better/worse. I felt my opinion needed to be shared.

lars512 13y ago

Even in "non-deterministic" style prolog, the execution order is still fixed, and dependencies are traversed in the order you wrote them. There will be no surprises. The missing full-stop is no different to a missing semi-colon, comma, or backslash, issues we still have in many modern languages.

calpaterson 13y ago
> The missing full-stop is no different to a missing semi-colon Yeah, it's an annoyance. The difference is that the feedback loop on operations codebases is longer, so they're more annoying.

char_pointer 13y ago

For Puppet you can use a validation script and a git commit hook to shorten the feedback loop on syntax errors: http://spoop.net/devops/2013/11/12/validate-puppet.html http://spoop.net/devops/2013/11/12/validate-puppet.html