5 ms·

Unpopular Opinion: IaC heavily benefits from not being "Actual Code". While DSL based IaC brings in additional constraints that developers hate dealing with, t

Unpopular Opinion: IaC heavily benefits from not being "Actual Code". While DSL based IaC brings in additional constraints that developers hate dealing with, those constraints very frequently map to constraints in the underlying resources and serve as more of a guardrail than a wall. I've unfortunately been involved in a number of projects where developers moved from "restrictive" dsl based tooling (puppet, ansible, terraform) to tooling that allowed more free-form code to leak in (chef, cdk, unholy in-house nonsense), and every time I've watched the infrastructure description go from something that could easily be tested and reasoned about to an awful mess nobody wants to touch for fear of breakage.

lucideer 4y ago
Counter Opinion: Well-meaning DSL constraints very quickly evolve into hackish defacto "actual code". Every. Time. I do understand the intent of a constrained DSL mapping to constraints of underlying resources, but more often than not those constraints are short-lived in the lifespan of the DSL, and quickly give way to it becoming a badly architected near-turing-complete programming language. Ansible (which you've listed) is a prime example: it's not YAML, it's a frankenstein of YAML + logic-heavy Jinja-hacks that really remove a lot of the constraints of pure declarative YAML. The only thing holding it back from becoming a horrible logic mess is very deliberate and well-marketed community convention.
- freedomben 4y ago
  Exactly. I've used Terraform, Ansible, Chef, and a couple of hybrids extensively and my current opinion is that for simple use cases a strict DSL is great, but as soon as you need to something higher (usually some sort of looping over a complex data structure and populating fields from those values) it gets real painful. Terraform is getting better at the loops and stuff but I still have a bunch of code from older versions of terraform that are erb templates that output terraform files, thus allowing to use Ruby functions. That was/is probably the least evil solution I've seen. The worst is when the infrastructure or devops team (or someone else) decides, "we need to abstract this away with our own tool."
  - danw1979 4y ago
    I mean, Terraform wasn’t even really an MVP for your typical large cloud deployments before 0.12, IMHO. I think those days of magic-HCL-generators are more-or-less behind us. All we need now is variables in backend config and delayed provider initialisation and we’re laughing !
  - naserbvqtery 4y ago
    > as soon as you need to something higher (usually some sort of looping over a complex data structure and populating fields from those values) it gets real painful I'd argue this is one such guardrail. Way more than once I've seen infrastructure resources exhausted (frequently cloud provider limits), because a hard line wasn't maintained between infrastructure code and application code. Frequently in the form of "It just makes sense for us to create one %s infrastructure resource for each %s in our application!" followed by a series of midnight conference calls weeks-months later.
- Arnavion 4y ago
  Yes, from personal experience, it starts with a DSL. Then someone notices it has a lot of boilerplate / repetition and writes a shell script to generate the DSL. Then people complain that remembering the shell script parameters is hard so someone writes a DSL as input for the shell script. Then...
octopoc 4y ago
> every time I've watched the infrastructure description go from something that could easily be tested and reasoned about to an awful mess nobody wants to touch for fear of breakage We programmers have been dealing with this kind of complexity for a long time, why not use some of the tools normally used to manage complexity in code? For example: 1. Comments 2. Code reviews 3. Gated merge requests etc. etc.
- naserbvqtery 4y ago
  Those things should all still be used with a dsl iac. They're not bringing anything new to the table in the argument of dsl vs general programming language for infrastructure. Arguably this comment even demonstrates a bit of what I consider one of the biggest problems in the space. It's common for someone who hasn't bought in to dsl based iac tooling to imply dsl based tooling is from a less sophisticated place where modern engineering practices aren't applied, and it's better to grow up and use big-boy tools. It's almost like they've forgotten the dsl based tooling was in fact built by other application developers with intense subject matter expertise in the space.
eternityforest 4y ago
Rarely ever have I seen something declarative and wished it was any other way. I think declarative should be the default, where practical, for pretty much anything. If you can't do it in declarative, it's often not a good idea anyway, or the declarative engine just needs an extension.

lucideer 4y ago

Counter Opinion: Well-meaning DSL constraints very quickly evolve into hackish defacto "actual code". Every. Time. I do understand the intent of a constrained DSL mapping to constraints of underlying resources, but more often than not those constraints are short-lived in the lifespan of the DSL, and quickly give way to it becoming a badly architected near-turing-complete programming language. Ansible (which you've listed) is a prime example: it's not YAML, it's a frankenstein of YAML + logic-heavy Jinja-hacks that really remove a lot of the constraints of pure declarative YAML. The only thing holding it back from becoming a horrible logic mess is very deliberate and well-marketed community convention.

freedomben 4y ago
Exactly. I've used Terraform, Ansible, Chef, and a couple of hybrids extensively and my current opinion is that for simple use cases a strict DSL is great, but as soon as you need to something higher (usually some sort of looping over a complex data structure and populating fields from those values) it gets real painful. Terraform is getting better at the loops and stuff but I still have a bunch of code from older versions of terraform that are erb templates that output terraform files, thus allowing to use Ruby functions. That was/is probably the least evil solution I've seen. The worst is when the infrastructure or devops team (or someone else) decides, "we need to abstract this away with our own tool."
- danw1979 4y ago
  I mean, Terraform wasn’t even really an MVP for your typical large cloud deployments before 0.12, IMHO. I think those days of magic-HCL-generators are more-or-less behind us. All we need now is variables in backend config and delayed provider initialisation and we’re laughing !
- naserbvqtery 4y ago
  > as soon as you need to something higher (usually some sort of looping over a complex data structure and populating fields from those values) it gets real painful I'd argue this is one such guardrail. Way more than once I've seen infrastructure resources exhausted (frequently cloud provider limits), because a hard line wasn't maintained between infrastructure code and application code. Frequently in the form of "It just makes sense for us to create one %s infrastructure resource for each %s in our application!" followed by a series of midnight conference calls weeks-months later.
Arnavion 4y ago
Yes, from personal experience, it starts with a DSL. Then someone notices it has a lot of boilerplate / repetition and writes a shell script to generate the DSL. Then people complain that remembering the shell script parameters is hard so someone writes a DSL as input for the shell script. Then...

octopoc 4y ago

> every time I've watched the infrastructure description go from something that could easily be tested and reasoned about to an awful mess nobody wants to touch for fear of breakage We programmers have been dealing with this kind of complexity for a long time, why not use some of the tools normally used to manage complexity in code? For example: 1. Comments 2. Code reviews 3. Gated merge requests etc. etc.

naserbvqtery 4y ago
Those things should all still be used with a dsl iac. They're not bringing anything new to the table in the argument of dsl vs general programming language for infrastructure. Arguably this comment even demonstrates a bit of what I consider one of the biggest problems in the space. It's common for someone who hasn't bought in to dsl based iac tooling to imply dsl based tooling is from a less sophisticated place where modern engineering practices aren't applied, and it's better to grow up and use big-boy tools. It's almost like they've forgotten the dsl based tooling was in fact built by other application developers with intense subject matter expertise in the space.

eternityforest 4y ago

Rarely ever have I seen something declarative and wished it was any other way. I think declarative should be the default, where practical, for pretty much anything. If you can't do it in declarative, it's often not a good idea anyway, or the declarative engine just needs an extension.