4 ms·

Agreed, I don't really see the upside of getting a "proper" programming language for this. I just got done with a client of mine migrating a considerable amoun

Agreed, I don't really see the upside of getting a "proper" programming language for this. I just got done with a client of mine migrating a considerable amount of infrastructure code that used Troposphere (Python -> CloudFormation) into Terraform 0.12. The old code was very difficult to reason about, had all sorts of inter-related components that caused infrastructure changes to be scary, and really there seemed to be no benefit of being able to write Python for it. They now have over 100 services in Terraform 0.12 and they feel confident even making network routing changes in prod during the middle of the day with the new system (something that was unheard of before). I've found it much easier to write declarative configs in Terraform over the years. Back when I was at Engine Yard we used a project called fog to write infrastructure code in Ruby. It was nice for the time (must have been 2012 or so). But again, I really don't think the problem is that I need a programming language to define my infrastructure. I need a way to declare my infrastructure and manage that state so that I know that what I declare is what is running. I totally can see teams choosing to write infrastructure code in TypeScript, and Go, and Python. And now you have a mess.

throwaway894345 7y ago
The idea is reuse. YAML (and probably Terraform, though I can't speak to it directly) doesn't give you many facilities for reusing blocks of config, especially if they vary subtly in some parameterized way. CloudFormation gives you some reusability in the way of nested-stacks and macros, but it's seriously heavy-handed. We do use Troposphere in a handful of cases, and it has its own problems, mostly in that it makes it hard to write declarative Python code with it (which is generally what you want--declarative code but with more expressive power than YAML). I have a prototype of an improvement to Troposphere that I built for my own amusement, and I think I'm on to something: https://github.com/weberc2/nimbus/blob/master/examples/src/nimbus_examples/s3bucket.py https://github.com/weberc2/nimbus/blob/master/examples/src/n... Note that this example is type-safe and declarative while Troposphere is not. Basically I don't think Troposphere is a good representation of what infrastructure-as-code(-not-yaml) could look like. Not sure about Pulumi as I haven't tried it. But I know the answer isn't YAML, it's not hacking an AST on top of YAML a la CloudFormation, it's not a different static dialect with its own dynamic hacks a la HCL, and it's not generating YAML with text templates a la Helm.
- mtalantikite 7y ago
  "The idea is reuse." That's what Terraform modules are for. If I need to bundle up things for, say, multiple environments, I can just write it as a module and configure it through the variables I've exposed. There's also a registry of community maintained modules if there are some common patterns I don't want to write on my own (https://registry.terraform.io/browse/modules https://registry.terraform.io/browse/modules).
  - throwaway894345 7y ago
    I can't speak to Terraform because I haven't used it, but I'm skeptical that modules are sufficient for many of the same reasons that CloudFormation templates are insufficient. I think at some level you need the ability to programmatically generate arbitrary static configuration, and you end up needing something very like a real programming language to do that. If you need something very like a real programming language, you should just use a real programming language instead of a tool that accidentally reinvents programming language concepts and the corresponding unnecessary learning curve that that implies for developers. I think the reason these tools are surviving in the "marketplace" is because they market is inexperienced and they conflate "generating static configs (to be passed into a tool that can apply the configs) with an imperative language" with "imperative provisioning of infrastructure". The market sees static/declarative solutions like Terraform and CloudFormation as the only alternatives to "imperative provisioning of infrastructure". I think as our industry gets more experience and tools for generating YAML/static-configs improve, it will be clear that these are the ways forward.
    - juliosueiras 7y ago
      want to point that ever since Terraform 0.12, all terraform params and modules etc are transformable in objects, what I mean by that is you can use functions like yamldecode or jsondecode to decode any yaml or json into terraform objects and pass into modules or resources or locals, also "real" programming language is always funny to me when ops are going to be much more comfortable with HCL. JSON/YAML are not in the same class as HCL. YAML maybe can do nesting with & but they can't just create simple config style interface, where the user facing side is a simple yaml file, and the terraform code underneath take it in and pass it to its modules

throwaway894345 7y ago

The idea is reuse. YAML (and probably Terraform, though I can't speak to it directly) doesn't give you many facilities for reusing blocks of config, especially if they vary subtly in some parameterized way. CloudFormation gives you some reusability in the way of nested-stacks and macros, but it's seriously heavy-handed. We do use Troposphere in a handful of cases, and it has its own problems, mostly in that it makes it hard to write declarative Python code with it (which is generally what you want--declarative code but with more expressive power than YAML). I have a prototype of an improvement to Troposphere that I built for my own amusement, and I think I'm on to something: https://github.com/weberc2/nimbus/blob/master/examples/src/nimbus_examples/s3bucket.py https://github.com/weberc2/nimbus/blob/master/examples/src/n... Note that this example is type-safe and declarative while Troposphere is not. Basically I don't think Troposphere is a good representation of what infrastructure-as-code(-not-yaml) could look like. Not sure about Pulumi as I haven't tried it. But I know the answer isn't YAML, it's not hacking an AST on top of YAML a la CloudFormation, it's not a different static dialect with its own dynamic hacks a la HCL, and it's not generating YAML with text templates a la Helm.

mtalantikite 7y ago
"The idea is reuse." That's what Terraform modules are for. If I need to bundle up things for, say, multiple environments, I can just write it as a module and configure it through the variables I've exposed. There's also a registry of community maintained modules if there are some common patterns I don't want to write on my own (https://registry.terraform.io/browse/modules https://registry.terraform.io/browse/modules).
- throwaway894345 7y ago
  I can't speak to Terraform because I haven't used it, but I'm skeptical that modules are sufficient for many of the same reasons that CloudFormation templates are insufficient. I think at some level you need the ability to programmatically generate arbitrary static configuration, and you end up needing something very like a real programming language to do that. If you need something very like a real programming language, you should just use a real programming language instead of a tool that accidentally reinvents programming language concepts and the corresponding unnecessary learning curve that that implies for developers. I think the reason these tools are surviving in the "marketplace" is because they market is inexperienced and they conflate "generating static configs (to be passed into a tool that can apply the configs) with an imperative language" with "imperative provisioning of infrastructure". The market sees static/declarative solutions like Terraform and CloudFormation as the only alternatives to "imperative provisioning of infrastructure". I think as our industry gets more experience and tools for generating YAML/static-configs improve, it will be clear that these are the ways forward.
  - juliosueiras 7y ago
    want to point that ever since Terraform 0.12, all terraform params and modules etc are transformable in objects, what I mean by that is you can use functions like yamldecode or jsondecode to decode any yaml or json into terraform objects and pass into modules or resources or locals, also "real" programming language is always funny to me when ops are going to be much more comfortable with HCL. JSON/YAML are not in the same class as HCL. YAML maybe can do nesting with & but they can't just create simple config style interface, where the user facing side is a simple yaml file, and the terraform code underneath take it in and pass it to its modules