9 ms·

What I find unfortunate about infrastructure-as-code tooling is that a lot of the tooling isn't actually using code, but instead uses esoteric configuration lan

What I find unfortunate about infrastructure-as-code tooling is that a lot of the tooling isn't actually using code, but instead uses esoteric configuration languages. Indeed, the article refers to Terraform with its custom syntax. Imho tools should use actual code (whether it's TypeScript or Kotlin or whatever) instead of reinventing constructs like loops and string interpolation. Thankfully these tools are getting more popular, because frankly I can't stand configuring another Kubernetes or GCP resource using a huge block of copy/pasted YAML.

robbyt 8y ago
This was called Chef, and every time I've encountered a Chef infrastructure it's been a complete mess.
amq 8y ago
Could you name some of those code-based tools?
- ryanworl 8y ago
  https://www.pulumi.com/ https://www.pulumi.com/
- closeparen 8y ago
  Start with hitting cloud provider APIs directly, then looking for value-add on top of that.
- cle 8y ago
  AWS CDK: https://github.com/awslabs/aws-cdk https://github.com/awslabs/aws-cdk
- rad_gruchalski 8y ago
  One of the oldest examples of this: https://github.com/infochimps-labs/ironfan https://github.com/infochimps-labs/ironfan. But yeah, Chef. Examples: https://github.com/infochimps-labs/ironfan-pantry/blob/master/cookbooks/volumes/README.md https://github.com/infochimps-labs/ironfan-pantry/blob/maste.... I used to use this heavily back in 2013, 2014. Infochimps got picked up by eBay, afair. Hence why this was never developed further.
argd678 8y ago
Agreed completely, the current state is like using a language with no debugger, no modern knowledge of code reuse, no real IDE integration, Terraform is especially poor on a large project since it’s parser doesn’t report line numbers on errors and it has no forward references or ways to resolve interdependence between objects and leaves that up to the user to manually converge its state, not to mention you have to rewrite everything for every cloud and it’s riddled with bugs for common uses. Pulumi is going down a better route in that they’re using a normal programing language with a normal tool set, however I’m skeptical of how their engine is designed with RPC calls for language bindings, since again it makes debugging more complex as opposed to just a normal sdk. They also don’t have debugging enabled yet.
lukehoban 8y ago
Here at Pulumi, this is something we are working hard on! For all the many benefits we've seen with Infrastructure as Code to date, the tools are still fairly primitive - copy/paste is the norm for reuse, testing is rare or non-existant, productivity during infrastructure development is low, continuous integration and delivery are largely ad-hoc, and there are very few higher-level libraries available to abstract away the complex details of today's modern cloud platforms. Net - it feels like we're still programming cloud infrastructure at the assembly-language level. I'm really excited about the opportunity to bring more software engineering rigor into the infrastructure as code space. At Pulumi we believe using existing programming languages is a key enabler of this. Pulumi is still a desired state model like other Infrastructure as Code offerings (so you can still preview changes and make minimal deltas to existing infrastructure) - but you can write code to construct that desired state. As a result, you get for loops and conditionals, you get types and error checking, you get IDE productivity tooling, you can create abstractions and interfaces around components, you can write tests, you can confidently refactor you infrastructure code, you can deliver and version components via robust package managers, and you can integrate naturally into CI/CD workflows. Pulumi isn't the only tool in this space - we're seeing things like Atomist bringing this same model to delivery pipelines, and AWS CDK bringing this model to the CloudFormation space. I'm excited about where these tools will take the Infrastructure as Code ecosystem in the coming years. [disclaimer - CTO at https://pulumi.io https://pulumi.io so clearly biased on this topic :-)]
- royjacobs 8y ago
  Pulumi definitely looks very nice and I hope you manage to get some traction in the k8s space. The amount of stacked tooling (like skaffold) currently being used in that space is papering over the real issue that raw YAML simply isn't working. So good luck to you!
hedwall 8y ago
What I've seen is that when you get a real programming language for IaC you'll end up with a half assed, poorly implemented and documented version of a declarative DSL that is organization specific. I saw it with early versions of Chef, I've seen it with Apache Auroras python based job specs for Mesos etc. I'd rather work around the limitations of a DSL that is declarative and limited but is consistent across orgs than having to retrain myself if I move to a new job.
- argd678 8y ago
  I think this points more to that it’s a normal project that’s developed and you need competent developers like on any software project. The DSL route was likely to dumb things down for operations people with little software experience, but in doing so they created declarative models that don’t model reality and have created such an awkward fit that it reduces productivity that’s a solved problem for decades now. Operations is the wrong place to do engineering, either that or operations needs to level up significantly since they do cause a lot of delays and damage to companies with these poorly design tools.
  - joshuamorton 8y ago
    using Turing complete languages for configs is a mistake. Configs are hard, and they're different to enough from normal software (for example DRY doesn't apply in the same circumstances) that using the same tools is a bad idea. I've seen what happens when swes use swe languages on configs. They get unintelligible. And then I have to clean up the messes. To be clear though, using Json or toml for configs is also often a mistake.
    - argd678 8y ago
      The problem is that the declarative model doesn’t fit, so in Terraform for example they’ve added branching and for loops even, so you’re making a language at that point. But rather than keeping the logic and the data structure separate, they’re mixed up with each other. This is a solved problem in many languages, Java especially. I think a valid question would be what is the right framework factoring or design should be, but pure declarative isn’t not and half declarative like Terraform are just repeating past mistakes.
deleted 8y ago
[deleted]
adontz 8y ago
I do not think imperative languages are better, because they suck at parallelization. And creating infrastructure elements in parallel is crucial. What I see as optimal is tool like make, which calls other command line tools. Resolves dependencies, processes errors. But not make, something with sane syntax and error handling. All these new fancy all-Go tools creep me out. Infrastructure tooling should not live in domain of one programming language. Extensibility should be language agnostic. If I want to write some Perl/Python/Bash script to support some very non standard part of my infrastructure, I should be able to. If I want to plug vendor specific utility execution into deployment pipeline, I should be able to. And it should be easy.
- mdaniel 8y ago
  > All these new fancy all-Go tools creep me out. It certainly makes one's ability to look into the machinery and know what is going on much harder. > If I want to write some Perl/Python/Bash script to support some very non standard part of my infrastructure, I should be able to That very feature is why I love ansible: if there is some quirk, or even an unreleased module (they only ship new features in major releases, I recently learned), then you can copy the upstream file, or a modified version of the existing one, or even a whole new module, into the `library` or `lookup_plugins` directory of your playbook and you're back in business. No fighting with golang anything. You can also write ansible modules in any language you like.
scarface74 8y ago
Wanting IAC to be procedural instead of declarative is like saying you want SQL to be procedural. You’re not telling AWS how to do something you’re telling it what you want the end state to be and let it figure out what needs to be created, updated, deleted, or replaced and the dependency chain what can be run in parallel when you create or update the template. There are linters and editors for CloudFormation that help you with autocomplete (?) and to warn you when you are specifying a a resource type. You can even add your own custom definitions to the linters for custom resource types that you create. CloudFormation just like generic SQL doesn’t by itself have the concept of loops. CloudFormation does support custom transforms and macros that you can create in any language and you can write programs that generate CF in many languages using the CDK where it will perform validation. I haven’t used it so I am being really hand wavy.
- johncolanduoni 8y ago
  I agree with this in general, but I think there’s room for more programmatic ways of calculating what the desired state is in tools like Terraform. There are already a lot of built-ins in terraform that are deterministic but procedural ways of calculating what the state is, without infecting the declarative nature of the reconciliation process.
  - nicois 8y ago
    We also were frustrated at the verboseness and vagaries of the terraform syntax. We ended up preprocessing them using jinja2, injecting our variables via its context, and using its cleaner syntax for expressing conditions, loops, etc. Now we have the best of both worlds.
    - pm90 8y ago
      ... until your jinja templating breaks for some obscure reason, or the underlying terraform resource upgrades (they follow the provider api after all), or if you need to do some state surgery and partition one terraform state into multiple ones, or until you want to import resources into the generated terraform or.... Apologies, I don't mean to be a downer. But... you're making your present life easier but making the code harder to maintain for future maintainers of your codebase.
kissgyorgy 8y ago
> Imho tools should use actual code (whether it's TypeScript or Kotlin or whatever) instead of reinventing constructs like loops and string interpolation. I disagree with this one. I believe describing an infrastructure should be fully declarative and the tool should decide how it needs to create resources based on the description. This way, the infrastructure code almost can't have bugs, but the tool can be fixed for everybody.
pknopf 8y ago
I don't mind using code to generate what the status should look like, as long as the code doesn't actually mutate the state. Similar to React and it's virtual dom.
aphextron 8y ago
>Imho tools should use actual code (whether it's TypeScript or Kotlin or whatever) instead of reinventing constructs like loops and string interpolation. The counterpoint to this is things like Gulp or Gradle which become a nightmare after a couple years of multiple developers and coding styles appending things here and there. Now rather than just spending a few hours learning a basic config DSL, I have to build up a mental execution model every time I want to add a build step.
nicodjimenez 8y ago
So true! Declarative tooling works great until you have a complex situation, and the abastraction leaks.
rhizome 8y ago
The way I think of it is that there's too much personality in this technology.
novaleaf 8y ago
I've moved my bash based GCP vm deployment script to typescript, via shelljs: https://www.npmjs.com/package/shelljs https://www.npmjs.com/package/shelljs ShellJs works pretty nice. Not only is my script now cross platform, but doing conditional logic and user prompting is a lot easier in code than bash. The only "issue" I've found is that ShellJs is quite barebones. I wrote a wrapper over it to do everything, such as nice question prompting and colored output.

robbyt 8y ago

This was called Chef, and every time I've encountered a Chef infrastructure it's been a complete mess.

amq 8y ago

Could you name some of those code-based tools?

ryanworl 8y ago
https://www.pulumi.com/ https://www.pulumi.com/
closeparen 8y ago
Start with hitting cloud provider APIs directly, then looking for value-add on top of that.
cle 8y ago
AWS CDK: https://github.com/awslabs/aws-cdk https://github.com/awslabs/aws-cdk
rad_gruchalski 8y ago
One of the oldest examples of this: https://github.com/infochimps-labs/ironfan https://github.com/infochimps-labs/ironfan. But yeah, Chef. Examples: https://github.com/infochimps-labs/ironfan-pantry/blob/master/cookbooks/volumes/README.md https://github.com/infochimps-labs/ironfan-pantry/blob/maste.... I used to use this heavily back in 2013, 2014. Infochimps got picked up by eBay, afair. Hence why this was never developed further.

argd678 8y ago

Agreed completely, the current state is like using a language with no debugger, no modern knowledge of code reuse, no real IDE integration, Terraform is especially poor on a large project since it’s parser doesn’t report line numbers on errors and it has no forward references or ways to resolve interdependence between objects and leaves that up to the user to manually converge its state, not to mention you have to rewrite everything for every cloud and it’s riddled with bugs for common uses. Pulumi is going down a better route in that they’re using a normal programing language with a normal tool set, however I’m skeptical of how their engine is designed with RPC calls for language bindings, since again it makes debugging more complex as opposed to just a normal sdk. They also don’t have debugging enabled yet.

lukehoban 8y ago

Here at Pulumi, this is something we are working hard on! For all the many benefits we've seen with Infrastructure as Code to date, the tools are still fairly primitive - copy/paste is the norm for reuse, testing is rare or non-existant, productivity during infrastructure development is low, continuous integration and delivery are largely ad-hoc, and there are very few higher-level libraries available to abstract away the complex details of today's modern cloud platforms. Net - it feels like we're still programming cloud infrastructure at the assembly-language level. I'm really excited about the opportunity to bring more software engineering rigor into the infrastructure as code space. At Pulumi we believe using existing programming languages is a key enabler of this. Pulumi is still a desired state model like other Infrastructure as Code offerings (so you can still preview changes and make minimal deltas to existing infrastructure) - but you can write code to construct that desired state. As a result, you get for loops and conditionals, you get types and error checking, you get IDE productivity tooling, you can create abstractions and interfaces around components, you can write tests, you can confidently refactor you infrastructure code, you can deliver and version components via robust package managers, and you can integrate naturally into CI/CD workflows. Pulumi isn't the only tool in this space - we're seeing things like Atomist bringing this same model to delivery pipelines, and AWS CDK bringing this model to the CloudFormation space. I'm excited about where these tools will take the Infrastructure as Code ecosystem in the coming years. [disclaimer - CTO at https://pulumi.io https://pulumi.io so clearly biased on this topic :-)]

royjacobs 8y ago
Pulumi definitely looks very nice and I hope you manage to get some traction in the k8s space. The amount of stacked tooling (like skaffold) currently being used in that space is papering over the real issue that raw YAML simply isn't working. So good luck to you!

hedwall 8y ago

What I've seen is that when you get a real programming language for IaC you'll end up with a half assed, poorly implemented and documented version of a declarative DSL that is organization specific. I saw it with early versions of Chef, I've seen it with Apache Auroras python based job specs for Mesos etc. I'd rather work around the limitations of a DSL that is declarative and limited but is consistent across orgs than having to retrain myself if I move to a new job.

argd678 8y ago
I think this points more to that it’s a normal project that’s developed and you need competent developers like on any software project. The DSL route was likely to dumb things down for operations people with little software experience, but in doing so they created declarative models that don’t model reality and have created such an awkward fit that it reduces productivity that’s a solved problem for decades now. Operations is the wrong place to do engineering, either that or operations needs to level up significantly since they do cause a lot of delays and damage to companies with these poorly design tools.
- joshuamorton 8y ago
  using Turing complete languages for configs is a mistake. Configs are hard, and they're different to enough from normal software (for example DRY doesn't apply in the same circumstances) that using the same tools is a bad idea. I've seen what happens when swes use swe languages on configs. They get unintelligible. And then I have to clean up the messes. To be clear though, using Json or toml for configs is also often a mistake.
  - argd678 8y ago
    The problem is that the declarative model doesn’t fit, so in Terraform for example they’ve added branching and for loops even, so you’re making a language at that point. But rather than keeping the logic and the data structure separate, they’re mixed up with each other. This is a solved problem in many languages, Java especially. I think a valid question would be what is the right framework factoring or design should be, but pure declarative isn’t not and half declarative like Terraform are just repeating past mistakes.

deleted 8y ago

[deleted]

adontz 8y ago

I do not think imperative languages are better, because they suck at parallelization. And creating infrastructure elements in parallel is crucial. What I see as optimal is tool like make, which calls other command line tools. Resolves dependencies, processes errors. But not make, something with sane syntax and error handling. All these new fancy all-Go tools creep me out. Infrastructure tooling should not live in domain of one programming language. Extensibility should be language agnostic. If I want to write some Perl/Python/Bash script to support some very non standard part of my infrastructure, I should be able to. If I want to plug vendor specific utility execution into deployment pipeline, I should be able to. And it should be easy.

mdaniel 8y ago
> All these new fancy all-Go tools creep me out. It certainly makes one's ability to look into the machinery and know what is going on much harder. > If I want to write some Perl/Python/Bash script to support some very non standard part of my infrastructure, I should be able to That very feature is why I love ansible: if there is some quirk, or even an unreleased module (they only ship new features in major releases, I recently learned), then you can copy the upstream file, or a modified version of the existing one, or even a whole new module, into the `library` or `lookup_plugins` directory of your playbook and you're back in business. No fighting with golang anything. You can also write ansible modules in any language you like.

scarface74 8y ago

Wanting IAC to be procedural instead of declarative is like saying you want SQL to be procedural. You’re not telling AWS how to do something you’re telling it what you want the end state to be and let it figure out what needs to be created, updated, deleted, or replaced and the dependency chain what can be run in parallel when you create or update the template. There are linters and editors for CloudFormation that help you with autocomplete (?) and to warn you when you are specifying a a resource type. You can even add your own custom definitions to the linters for custom resource types that you create. CloudFormation just like generic SQL doesn’t by itself have the concept of loops. CloudFormation does support custom transforms and macros that you can create in any language and you can write programs that generate CF in many languages using the CDK where it will perform validation. I haven’t used it so I am being really hand wavy.

johncolanduoni 8y ago
I agree with this in general, but I think there’s room for more programmatic ways of calculating what the desired state is in tools like Terraform. There are already a lot of built-ins in terraform that are deterministic but procedural ways of calculating what the state is, without infecting the declarative nature of the reconciliation process.
- nicois 8y ago
  We also were frustrated at the verboseness and vagaries of the terraform syntax. We ended up preprocessing them using jinja2, injecting our variables via its context, and using its cleaner syntax for expressing conditions, loops, etc. Now we have the best of both worlds.
  - pm90 8y ago
    ... until your jinja templating breaks for some obscure reason, or the underlying terraform resource upgrades (they follow the provider api after all), or if you need to do some state surgery and partition one terraform state into multiple ones, or until you want to import resources into the generated terraform or.... Apologies, I don't mean to be a downer. But... you're making your present life easier but making the code harder to maintain for future maintainers of your codebase.

kissgyorgy 8y ago

> Imho tools should use actual code (whether it's TypeScript or Kotlin or whatever) instead of reinventing constructs like loops and string interpolation. I disagree with this one. I believe describing an infrastructure should be fully declarative and the tool should decide how it needs to create resources based on the description. This way, the infrastructure code almost can't have bugs, but the tool can be fixed for everybody.

pknopf 8y ago

I don't mind using code to generate what the status should look like, as long as the code doesn't actually mutate the state. Similar to React and it's virtual dom.

aphextron 8y ago

>Imho tools should use actual code (whether it's TypeScript or Kotlin or whatever) instead of reinventing constructs like loops and string interpolation. The counterpoint to this is things like Gulp or Gradle which become a nightmare after a couple years of multiple developers and coding styles appending things here and there. Now rather than just spending a few hours learning a basic config DSL, I have to build up a mental execution model every time I want to add a build step.

nicodjimenez 8y ago

So true! Declarative tooling works great until you have a complex situation, and the abastraction leaks.

rhizome 8y ago

The way I think of it is that there's too much personality in this technology.

novaleaf 8y ago

I've moved my bash based GCP vm deployment script to typescript, via shelljs: https://www.npmjs.com/package/shelljs https://www.npmjs.com/package/shelljs ShellJs works pretty nice. Not only is my script now cross platform, but doing conditional logic and user prompting is a lot easier in code than bash. The only "issue" I've found is that ShellJs is quite barebones. I wrote a wrapper over it to do everything, such as nice question prompting and colored output.