4 ms·

Wisdom for ansible would be to ditch the dsl and expose a pure python declarative api, with imperative escape hatches and a set of best practices + conventions

Wisdom for ansible would be to ditch the dsl and expose a pure python declarative api, with imperative escape hatches and a set of best practices + conventions encouraged by templates anf linters. Instead, we get a terrible dsl baked in a bad markup language that is hard to write, painful to debug and impossible to maintain. That you have to come up with concepts like an inventory, fact and magic variables should have been your first clue that they were reinventing badly what any language ecosystems provides for free. And the worst is, ansible is probably one if the best deployment systems at our disposal.

drcongo 4y ago
I've been very slowly rebuilding all our Ansible roles in PyInfra for that exact reason.
- bongobingo1 4y ago
  Also been going down this path. pyinfra is overlooked I think. It's not perfect, I still feel like I'm dropping down to just hitting `server.shell` a bunch , but at least as a programmer first, it feels more comfortable vs the mountains of yaml (and doc searching) for Ansible. On the whole it's a less frustrating experience. I still have a vague feeling that in my case - where each server(s) is generally customised per client and I've never felt comfortable just pointing ansible (or pyinfra) and a configured server and re-running it - that perhaps I would really just be better off with a bunch of shell scripts. I don't generally tear-down and throw-up multiple servers a day, week or even month. rrconf sort of fills this but works off a bit more infrastructure than I want to manage with its different git repos per action. [1] https://github.com/rrconf/rrconf https://github.com/rrconf/rrconf
cersa8 4y ago
Pulumi could have filled this nicely but only supports a few cloud providers.
candiddevmike 4y ago
That's a hot take. In my experience Ansible is trivial to maintain and, due to the lack of better templating like Salt, ensures most playbooks look relatively similar as it's meant to be a simple, straight forward task runner. Onboarding folks into Ansible is a breeze as everything matches the standard (roles, tasks, group_vars, etc). It's dumb, reliable, and does exactly what it says on the tin. My biggest complaint with Ansible is the execution aspect. The fact that mitogen hasn't become the standard strategy plugin and instead Ansible still opens an SSH connection per task is nuts.
- everforward 4y ago
  My experience is that Ansible becomes exponentially more difficult with the difficulty of your workflows, where programmatic tends to be logarithmic. For low complexity tasks, Ansible is simpler and for high complexity tasks it gets more complicated. Just as a random example, for complex group structures you have to know/remember that although Ansible allows nesting groups, groups are actually a flat structure. If you have prod.webservers and uat.webservers, the 'webservers' group will include members of both. That's not hard to work around, but it can easily lead to situations where you end up deploying to prod even though you only intended to deploy to UAT. Variables are kind of a mess too. Assuming the docs are authoritative, there are 22 different places that variables can come from and which one is used will depend on the precedence. At some point, a 'switch' or series of 'if-elif' statements start to sound much simpler.
  - candiddevmike 4y ago
    RE: variables, I'm a heathen and set `hash_behaviour = merge`. This forces nested dict values to be merged instead of overwritten (lists cannot be merged, unlike Salt...). For whatever reason Ansible encourages users to not use this [1], which makes variable usage far more difficult than it should be IMO and leads to situations like you describe where folks have various top level variables for overrides (ssh:, ssh_local:, ssh_global:, etc to combine values). [1] https://docs.ansible.com/ansible/latest/reference_appendices/config.html#default-hash-behaviour https://docs.ansible.com/ansible/latest/reference_appendices...
pid-1 4y ago
Yeah after using CloudFormation, Terraform, Pulumi, AWS CDK and k8s + YAML, I'm convinced using a real programing language idiomatically is the correct way of doing IaC. Many languages totally support declarative idioms out of the box (like you mention), there's no need to reinvent anything here. In particular I'm a big fan of AWS CDK, although it only works for AWS. (Yes, I know about CDKTF, but it's buggy alpha software).
goodpoint 4y ago
Wisdom for ansible would be to ditch ansible and any similar tool.
Octabrain 4y ago
> ditch the dsl and expose a pure python declarative api, with imperative escape hatches and a set of best practices + conventions encouraged by templates anf linters. Fully wholeheartedly agree but, as I stated in previous ocasions in this site, after the nightmares I've seen people, in many different places, doing with a high level tool like Ansible (and also Terraform), I apply heuristics about the probable outcome of every DevOps using a proper Turing-complete language as a general trend and I just want to leave CS and buy a hut in the middle of nowhere. Perhaps something like that must come after a shift in the mentality of the whole business where people in CS in general, assumes one needs to be a good coder, no matter if you do DevOps of software and therefore, learn about software design patterns, reusability and best coding practices.
- Spivak 4y ago
  The issue is that Ansible right now today is just Python but if you tied chopsticks to your fingers when typing. It’s not declarative, it’s sequentially evaluated with all the flow control and concurrency bugs to go with it. You have to already be a good coder to write Ansible but you have all your limbs tied behind your back because you get none of the features of Python to help you reign in complexity and validate correctness.
  - zzzeek 4y ago
    The killer feature of Ansible vs. Python code is, noting I have a ton of both here for infrastructure related tasks (as well as shell, why not), when you have it running for three years on some box somewhere and something breaks, you go look at it for the first time in three years and it makes sense. My Python infrastructure tools OTOH, while they are great, if something goes wrong with them I have to spend half a day reading the sourcecode and remembering what their particular architcture / conventions are. it's like the dumbness of Ansible is really the main feature it has.
    - Spivak 4y ago
      I agree with you but I think that dumb + size => complicated which is where I live. When you have a playbook that is thousands of tasks long the dumbness leads to exactly the problem you have with Python.
traverseda 4y ago
https://github.com/Fizzadar/pyinfra https://github.com/Fizzadar/pyinfra
xorcist 4y ago
I understand where you are coming from. It's interesting to see the Puppet/Chef-debate played out over and over again. You generally want an overarching single source of truth for IT systems. One that defined which parts a system contains and how it is set up. This starts to get important when you want maintainable testing environments, and more so as the number of system integration points grow, but it generally makes tracking changes and having a controlled deployment process so much easier. So it is natural to start with a declaration of how small pieces fit together. This is Puppet/Ansible where the language is just a list of objects, where each object is implemented in "real" programming language. But the list soon needs logic, if not for other reasons than because testing environments have subtly different requirements, and the list grows into a semi-language. That's not enough and people new to the system soon feels constrained, why not ditch that domain specific thing and implement the whole thing in the "real" language? This is Chef/Salt. But now all you have is another object, which needs an overarching list of objects, and soon we have another simple description for that. And then we repeat this all over again. (There are specific issues for some parts of this, which is not part of the above story. Ansible didn't have to choose yaml for their domain specific language, they should have seen that coming. But that's not for this story. We instead get to see this repeated again in the container space with templates and Helm charts. Also Jenkins pipelines which slowly mutates to Groovy and back.)

drcongo 4y ago

I've been very slowly rebuilding all our Ansible roles in PyInfra for that exact reason.

bongobingo1 4y ago
Also been going down this path. pyinfra is overlooked I think. It's not perfect, I still feel like I'm dropping down to just hitting `server.shell` a bunch , but at least as a programmer first, it feels more comfortable vs the mountains of yaml (and doc searching) for Ansible. On the whole it's a less frustrating experience. I still have a vague feeling that in my case - where each server(s) is generally customised per client and I've never felt comfortable just pointing ansible (or pyinfra) and a configured server and re-running it - that perhaps I would really just be better off with a bunch of shell scripts. I don't generally tear-down and throw-up multiple servers a day, week or even month. rrconf sort of fills this but works off a bit more infrastructure than I want to manage with its different git repos per action. [1] https://github.com/rrconf/rrconf https://github.com/rrconf/rrconf

cersa8 4y ago

Pulumi could have filled this nicely but only supports a few cloud providers.

candiddevmike 4y ago

That's a hot take. In my experience Ansible is trivial to maintain and, due to the lack of better templating like Salt, ensures most playbooks look relatively similar as it's meant to be a simple, straight forward task runner. Onboarding folks into Ansible is a breeze as everything matches the standard (roles, tasks, group_vars, etc). It's dumb, reliable, and does exactly what it says on the tin. My biggest complaint with Ansible is the execution aspect. The fact that mitogen hasn't become the standard strategy plugin and instead Ansible still opens an SSH connection per task is nuts.

everforward 4y ago
My experience is that Ansible becomes exponentially more difficult with the difficulty of your workflows, where programmatic tends to be logarithmic. For low complexity tasks, Ansible is simpler and for high complexity tasks it gets more complicated. Just as a random example, for complex group structures you have to know/remember that although Ansible allows nesting groups, groups are actually a flat structure. If you have prod.webservers and uat.webservers, the 'webservers' group will include members of both. That's not hard to work around, but it can easily lead to situations where you end up deploying to prod even though you only intended to deploy to UAT. Variables are kind of a mess too. Assuming the docs are authoritative, there are 22 different places that variables can come from and which one is used will depend on the precedence. At some point, a 'switch' or series of 'if-elif' statements start to sound much simpler.
- candiddevmike 4y ago
  RE: variables, I'm a heathen and set `hash_behaviour = merge`. This forces nested dict values to be merged instead of overwritten (lists cannot be merged, unlike Salt...). For whatever reason Ansible encourages users to not use this [1], which makes variable usage far more difficult than it should be IMO and leads to situations like you describe where folks have various top level variables for overrides (ssh:, ssh_local:, ssh_global:, etc to combine values). [1] https://docs.ansible.com/ansible/latest/reference_appendices/config.html#default-hash-behaviour https://docs.ansible.com/ansible/latest/reference_appendices...

pid-1 4y ago

Yeah after using CloudFormation, Terraform, Pulumi, AWS CDK and k8s + YAML, I'm convinced using a real programing language idiomatically is the correct way of doing IaC. Many languages totally support declarative idioms out of the box (like you mention), there's no need to reinvent anything here. In particular I'm a big fan of AWS CDK, although it only works for AWS. (Yes, I know about CDKTF, but it's buggy alpha software).

goodpoint 4y ago

Wisdom for ansible would be to ditch ansible and any similar tool.

Octabrain 4y ago

> ditch the dsl and expose a pure python declarative api, with imperative escape hatches and a set of best practices + conventions encouraged by templates anf linters. Fully wholeheartedly agree but, as I stated in previous ocasions in this site, after the nightmares I've seen people, in many different places, doing with a high level tool like Ansible (and also Terraform), I apply heuristics about the probable outcome of every DevOps using a proper Turing-complete language as a general trend and I just want to leave CS and buy a hut in the middle of nowhere. Perhaps something like that must come after a shift in the mentality of the whole business where people in CS in general, assumes one needs to be a good coder, no matter if you do DevOps of software and therefore, learn about software design patterns, reusability and best coding practices.

Spivak 4y ago
The issue is that Ansible right now today is just Python but if you tied chopsticks to your fingers when typing. It’s not declarative, it’s sequentially evaluated with all the flow control and concurrency bugs to go with it. You have to already be a good coder to write Ansible but you have all your limbs tied behind your back because you get none of the features of Python to help you reign in complexity and validate correctness.
- zzzeek 4y ago
  The killer feature of Ansible vs. Python code is, noting I have a ton of both here for infrastructure related tasks (as well as shell, why not), when you have it running for three years on some box somewhere and something breaks, you go look at it for the first time in three years and it makes sense. My Python infrastructure tools OTOH, while they are great, if something goes wrong with them I have to spend half a day reading the sourcecode and remembering what their particular architcture / conventions are. it's like the dumbness of Ansible is really the main feature it has.
  - Spivak 4y ago
    I agree with you but I think that dumb + size => complicated which is where I live. When you have a playbook that is thousands of tasks long the dumbness leads to exactly the problem you have with Python.

traverseda 4y ago

https://github.com/Fizzadar/pyinfra https://github.com/Fizzadar/pyinfra

xorcist 4y ago

I understand where you are coming from. It's interesting to see the Puppet/Chef-debate played out over and over again. You generally want an overarching single source of truth for IT systems. One that defined which parts a system contains and how it is set up. This starts to get important when you want maintainable testing environments, and more so as the number of system integration points grow, but it generally makes tracking changes and having a controlled deployment process so much easier. So it is natural to start with a declaration of how small pieces fit together. This is Puppet/Ansible where the language is just a list of objects, where each object is implemented in "real" programming language. But the list soon needs logic, if not for other reasons than because testing environments have subtly different requirements, and the list grows into a semi-language. That's not enough and people new to the system soon feels constrained, why not ditch that domain specific thing and implement the whole thing in the "real" language? This is Chef/Salt. But now all you have is another object, which needs an overarching list of objects, and soon we have another simple description for that. And then we repeat this all over again. (There are specific issues for some parts of this, which is not part of the above story. Ansible didn't have to choose yaml for their domain specific language, they should have seen that coming. But that's not for this story. We instead get to see this repeated again in the container space with templates and Helm charts. Also Jenkins pipelines which slowly mutates to Groovy and back.)