7 ms·

I wish there was a server provisioning "library" written in an ML-ish language. The trouble with even a simple, well-made tool like Ansible is its dynamic DSL.

by jasim 9y ago

I wish there was a server provisioning "library" written in an ML-ish language. The trouble with even a simple, well-made tool like Ansible is its dynamic DSL. The YAML file is essentially code, but modules like "file" and "apt" written as configuration entries give a declarative feel to it. But they are just procedures that strive to be idempotent, and its YAML file is just a series of procedure calls, sprinkled with some facility for organizing hierarchies and sharing code between them. YAML the format is also not a great way to encode systems data. Thankfully Ansible has documented the gotchas with its space-indentation and need for explicit quoting in http://docs.ansible.com/ansible/latest/YAMLSyntax.html#gotchas http://docs.ansible.com/ansible/latest/YAMLSyntax.html#gotch.... But that is just an attempt to band-aid over the unsuitability of YAML for this purpose. Chef's configuration on the other hand can be done in Ruby, but you're still constrained to its DSLs. The shape and structure of Chef's objects can only gleaned by reading reams of documentation, and worse, by running the cookbooks with debug statements. The problem is compounded when you upgrade these tools and things stop working silently. The community solution for this unsurprisingly is unit tests. I'm still grappling with that idea. But what if we could write our own provisioning code in Haskell or OCaml/Reason, with library functions that help install packages, returns errors in neat sum types which we can confidently handle and doesn't leave us to worry about the known unknowns that we haven't enumerated and handled, and a compiler with exhaustive pattern matching that helpfully tells us about all the record types and their structure within a fast edit-compile cycle, ...

crdoconnor 9y ago
>The YAML file is essentially code YAML is not turing complete and is hence not "code". That is a large part of what makes ansible such a powerful tool - it enforces a strict separation of concerns between the definition of server configuration and its execution and keeps one in a language that is deliberately not able to be made more complex than is needed, is thus kept readable and straightforward to maintain. By contrast, turing complete languages are veritable magnets for technical debt, which is why a bad chef playbook will always be worse than a bad ansible playbook. Haskell, being turing complete, would suffer the same issue albeit not to the same degree as ruby. >YAML the format is also not a great way to encode systems data. Thankfully Ansible has documented the gotchas with its space-indentation and need for explicit quoting in http://docs.ansible.com/ansible/latest/YAMLSyntax.html#gotch... http://docs.ansible.com/ansible/latest/YAMLSyntax.html#gotch.... But that is just an attempt to band-aid over the unsuitability of YAML for this purpose. YAML has a few oblique gotchas but they don't affect the fundamental suitability of it for this purpose. Once you get used to it, space indentation becomes a feature, not a bug.
- emmelaich 9y ago
  I agree with most of your comment but > YAML is not turing complete and is hence not "code". seems wrong to me. In fact I've heard code/coding to include non-Turing complete languages (e.g. html) as distinct from programs/programming.
  - guywhocodes 9y ago
    I think SQL is another prime example, being able to solve a problem effectively without resorting to Turing-completeness is preferable to me, as long as that language is simple enough.
    - gshakir 9y ago
      Yes agreed, Declarative languages like SQL or Ansible are great for most of the use cases. You can always use extensions to get down to imperative steps.
  - DerBesserWisser 9y ago
    HTML is a markup language and not a programming language. IMHO you can only "code" in programming languages (not necesseraly executed by a CPU). A programming language does not need to be turing complete though to "code" in it. You're just limited to the problems you can solve with a turing-incomplete language. It's interesting to think if using a problem-declaration language is coding.
- jasim 9y ago
  Any sufficiently complicated configuration language contains an ad hoc, informally-specified, bug-ridden, slow implementation of a Turing complete programming language. Ansible playbooks have variables, loops, conditionals, scoping rules, inheritance, and you can even extend it by writing custom modules in Python, which is a Turing complete language. The problem with not using a real programming language when you're trying to essentially "program" is that you're expressing complex programs without any of the abstractions and tooling support that a real language has. Despite all that I like Ansible's constrained format and the resulting simplicity compared to Chef. But I think we can take those design aesthetics and apply it in a statically typed library and maybe get something more sturdy.
emmelaich 9y ago
Yeah, me too (a stronger / more statically) typed language. But you're up against a potentially changing state -- the file system. So that only gets you so far. Idempotency is very hard to achieve. What gets me is there is an excellent tool -- rsync -- which is extremely fast and idempotent. The various config tools make far too little use of it.
- deleted 9y ago
  [deleted]
arianvanp 9y ago
This exists! and it's pretty awesome to use! https://hackage.haskell.org/package/propellor https://hackage.haskell.org/package/propellor
- jasim 9y ago
  You wonderful person! "Properties are defined using Haskell in the file ~/.propellor/config.hs. There is fairly complete API documentation, which includes many built-in Properties for dealing with Apt and Apache , Cron and Commands , Dns and Docker, etc. There is no special language as used in puppet, chef, ansible, etc.. just the full power of Haskell. Hopefully that power can be put to good use in making declarative properties that are powerful, nicely idempotent, and easy to adapt to a system's special needs!" I'll be learning Haskell for real this time.
susi22 9y ago
I'm a Clojure dev so I really like fast edit-compile cycles. Note, Ansible will happily take in JSON, so you can take your favorite language and write all the playbooks in that language. Just generate the playbooks on the fly (ie. genearte JSON not yaml) and shell out to ansible to run it. So you have all the features of ansible available but can arbitrarily build up your playbooks in your favorite language. It's hacky code, but that's exactly what I did in Clojure: https://gist.github.com/rauhs/a72cfbeef4c80f9a58480484c49e7a51#file-example-clj-L124-L257 https://gist.github.com/rauhs/a72cfbeef4c80f9a58480484c49e7a... With a normal REPL this works well especially when developing plays.
- emmelaich 9y ago
  > Ansible will happily take in JSON Yes, JSON is very close to a proper subset of Yaml.
gtirloni 9y ago
This isn't a replacement for configuration management but allows you to directly program into code how the application will be deployed. It seems interesting. https://metaparticle.io/about https://metaparticle.io/about
vesak 9y ago
Not exactly the same, but http://nixos.org http://nixos.org would give you some of what you want.
drcongo 9y ago
I came to Ansible from Puppet and Chef because in the early days, it was way easier to get going. But with almost every release it gets worse, and so much of this is down to the YML / Jinja hybrid DSL. We have a fantastically powerful fabfile (Python/Fabric) that we include in every project, it keeps getting more powerful over time because anything we need it to do we can just write the code to do it. Our Ansible roles are a real bind to maintain though. I'll definitely take a look at Propellor recommended below, thanks.
- xref 9y ago
  hasn't fabric been abandonware on python 2 for years?
  - drcongo 9y ago
    Sort of, there's a Python 3 fork of it though which is solid.
gshakir 9y ago
I don't wish that. How many DevOps people you know that can grok Haskell? There is a reason why Ansible is so popular, the barrier to entry is so low and that's the way it should be.
breatheoften 9y ago
As a former ansible user -- I'm completely there. There are lot of things to like about ansible but the unwieldy ad-hoc non-composable task description language and the lack of static verification for anything makes the whole experience so much worse than what I think is possible in this space. Ansible has a 'fact gathering' stage which is the first thing that happens after connecting to a host -- I've been dreaming about a system that is capable of leveraging an execution model like ansible's for 'gathering facts' from a set of hosts -- and then turning those facts into statically analyzable types in the context of some (statically typed) language -- with convenient language-level/library-level interfaces for expressing common orchestration/system-admin/stateful-scripting logic. I think such a system would be _EPIC_ to use.
- emmelaich 9y ago
  Facebook's osquery could help here. You just(1) have to implement update functions as well. (1) that's the "just" that doesn't mean "just" :-)
- orev 9y ago
  What have you moved to if no longer using Ansible?

by jasim 9y ago

crdoconnor 9y ago

>The YAML file is essentially code YAML is not turing complete and is hence not "code". That is a large part of what makes ansible such a powerful tool - it enforces a strict separation of concerns between the definition of server configuration and its execution and keeps one in a language that is deliberately not able to be made more complex than is needed, is thus kept readable and straightforward to maintain. By contrast, turing complete languages are veritable magnets for technical debt, which is why a bad chef playbook will always be worse than a bad ansible playbook. Haskell, being turing complete, would suffer the same issue albeit not to the same degree as ruby. >YAML the format is also not a great way to encode systems data. Thankfully Ansible has documented the gotchas with its space-indentation and need for explicit quoting in http://docs.ansible.com/ansible/latest/YAMLSyntax.html#gotch... http://docs.ansible.com/ansible/latest/YAMLSyntax.html#gotch.... But that is just an attempt to band-aid over the unsuitability of YAML for this purpose. YAML has a few oblique gotchas but they don't affect the fundamental suitability of it for this purpose. Once you get used to it, space indentation becomes a feature, not a bug.

emmelaich 9y ago
I agree with most of your comment but > YAML is not turing complete and is hence not "code". seems wrong to me. In fact I've heard code/coding to include non-Turing complete languages (e.g. html) as distinct from programs/programming.
- guywhocodes 9y ago
  I think SQL is another prime example, being able to solve a problem effectively without resorting to Turing-completeness is preferable to me, as long as that language is simple enough.
  - gshakir 9y ago
    Yes agreed, Declarative languages like SQL or Ansible are great for most of the use cases. You can always use extensions to get down to imperative steps.
- DerBesserWisser 9y ago
  HTML is a markup language and not a programming language. IMHO you can only "code" in programming languages (not necesseraly executed by a CPU). A programming language does not need to be turing complete though to "code" in it. You're just limited to the problems you can solve with a turing-incomplete language. It's interesting to think if using a problem-declaration language is coding.
jasim 9y ago
Any sufficiently complicated configuration language contains an ad hoc, informally-specified, bug-ridden, slow implementation of a Turing complete programming language. Ansible playbooks have variables, loops, conditionals, scoping rules, inheritance, and you can even extend it by writing custom modules in Python, which is a Turing complete language. The problem with not using a real programming language when you're trying to essentially "program" is that you're expressing complex programs without any of the abstractions and tooling support that a real language has. Despite all that I like Ansible's constrained format and the resulting simplicity compared to Chef. But I think we can take those design aesthetics and apply it in a statically typed library and maybe get something more sturdy.

emmelaich 9y ago

Yeah, me too (a stronger / more statically) typed language. But you're up against a potentially changing state -- the file system. So that only gets you so far. Idempotency is very hard to achieve. What gets me is there is an excellent tool -- rsync -- which is extremely fast and idempotent. The various config tools make far too little use of it.

deleted 9y ago
[deleted]

arianvanp 9y ago

This exists! and it's pretty awesome to use! https://hackage.haskell.org/package/propellor https://hackage.haskell.org/package/propellor

jasim 9y ago
You wonderful person! "Properties are defined using Haskell in the file ~/.propellor/config.hs. There is fairly complete API documentation, which includes many built-in Properties for dealing with Apt and Apache , Cron and Commands , Dns and Docker, etc. There is no special language as used in puppet, chef, ansible, etc.. just the full power of Haskell. Hopefully that power can be put to good use in making declarative properties that are powerful, nicely idempotent, and easy to adapt to a system's special needs!" I'll be learning Haskell for real this time.

susi22 9y ago

I'm a Clojure dev so I really like fast edit-compile cycles. Note, Ansible will happily take in JSON, so you can take your favorite language and write all the playbooks in that language. Just generate the playbooks on the fly (ie. genearte JSON not yaml) and shell out to ansible to run it. So you have all the features of ansible available but can arbitrarily build up your playbooks in your favorite language. It's hacky code, but that's exactly what I did in Clojure: https://gist.github.com/rauhs/a72cfbeef4c80f9a58480484c49e7a51#file-example-clj-L124-L257 https://gist.github.com/rauhs/a72cfbeef4c80f9a58480484c49e7a... With a normal REPL this works well especially when developing plays.

emmelaich 9y ago
> Ansible will happily take in JSON Yes, JSON is very close to a proper subset of Yaml.

gtirloni 9y ago

This isn't a replacement for configuration management but allows you to directly program into code how the application will be deployed. It seems interesting. https://metaparticle.io/about https://metaparticle.io/about

vesak 9y ago

Not exactly the same, but http://nixos.org http://nixos.org would give you some of what you want.

drcongo 9y ago

I came to Ansible from Puppet and Chef because in the early days, it was way easier to get going. But with almost every release it gets worse, and so much of this is down to the YML / Jinja hybrid DSL. We have a fantastically powerful fabfile (Python/Fabric) that we include in every project, it keeps getting more powerful over time because anything we need it to do we can just write the code to do it. Our Ansible roles are a real bind to maintain though. I'll definitely take a look at Propellor recommended below, thanks.

xref 9y ago
hasn't fabric been abandonware on python 2 for years?
- drcongo 9y ago
  Sort of, there's a Python 3 fork of it though which is solid.

gshakir 9y ago

I don't wish that. How many DevOps people you know that can grok Haskell? There is a reason why Ansible is so popular, the barrier to entry is so low and that's the way it should be.

breatheoften 9y ago

As a former ansible user -- I'm completely there. There are lot of things to like about ansible but the unwieldy ad-hoc non-composable task description language and the lack of static verification for anything makes the whole experience so much worse than what I think is possible in this space. Ansible has a 'fact gathering' stage which is the first thing that happens after connecting to a host -- I've been dreaming about a system that is capable of leveraging an execution model like ansible's for 'gathering facts' from a set of hosts -- and then turning those facts into statically analyzable types in the context of some (statically typed) language -- with convenient language-level/library-level interfaces for expressing common orchestration/system-admin/stateful-scripting logic. I think such a system would be _EPIC_ to use.

emmelaich 9y ago
Facebook's osquery could help here. You just(1) have to implement update functions as well. (1) that's the "just" that doesn't mean "just" :-)
orev 9y ago
What have you moved to if no longer using Ansible?