4 ms·
You have to understand that when IaC was new, the marketing was “it’s so simple you can just write YAML/JSON/etc” because frankly the industry was too dumb to u
by throwaway894345 5y ago
You have to understand that when IaC was new, the marketing was “it’s so simple you can just write YAML/JSON/etc” because frankly the industry was too dumb to understand that “using a real programming language to generate a description of the desired resource state” and “using a real programming language to imperatively reconcile the current and desired states oneself” are different things. So Terraform began with something that resembled YAML in its static-ness, and over time, more power was required so they would bolt on a dynamic feature but were reluctant to give the impression that they were building a programming language so the feature would be as obscure as possible. But that wouldn’t be enough either so they would add still more dynamic features, each comparably obscure until in time they’d built a complete, obscure programming language.
But this wasn’t just Terraform! The entire industry did this too. CloudFormation began as simple JSON, but over time they allowed you to encode the abstract syntax tree of a shitty programming language in your YAML, and CloudFormation would interpret it. However stupid that may sound, in the Kubernetes world, we have Helm which lets you generate YAML with text templates which is honestly the dumbest idea in the world (imagine a compiler that generates syntactically invalid machine code if the input program has an extra white space character).
Of course in all of these cases the answer is staring us in the face: use a static language (YAML, JSON, etc) to describe the desired state, and use a higher level language (like Python or Starlark or Dhall or etc) to generate that static desired state description. The only thing Terraform (or any IaC tool) should care about is the YAML description. That it is generated from Starlark or TypeScript is just an implementation detail.
Instead of that, though, we get CDKs which are so close, but admittedly I haven’t used them in anger yet.
- x3n0ph3n3 5y agoOne of the best parts of CloudFormation was their introduction of Macros. You can take either your whole template or just a snippet, and perform dynamic transformations by calling a lambda. I'll admit I've gone so far as being able to embed ERB (Ruby) into my templates in order to more dynamically define some resources based on stack parameters. I can also create N resources with common configuration based on the values of a CommaDelimitedList.
- throwaway894345 5y agoI think the idea here is that macros are neat in any language, but in CloudFormation they can help automate stuff that is only difficult because of CloudFormation, and the macros themselves are harder to use than those in a normal programming language. In all cases, I think it’s strictly less nice than generating your CloudFormation YAML with Python or similar.
- x3n0ph3n3 5y agoThe problem of generating your own YAML is you end up having to maintain multiple copies of nearly identical templates and keeping them in S3. I have done a bit of that and maintaining those as build-time assets rather than run-time assets is less appealing. Granted, that's required whenever you get into the realm of dynamically-determined sets of parameters.
- throwaway894345 5y agoYes, this is another problem with cloudformation, although IIRC I just used “aws cloudformation deploy” and let it manage my s3 resources.
- bopbeepboop 5y agoThe horrible JSON ASTs serve a point, and CDK makes them beautiful because it automatically generates them for you. I have a proxy Lambda and it has references to a bunch of SQS queues by names, and instead of managing that myself, I just: queue_dict = {} for name, queue in my_queues: queue_dict[name] = queue.arn queue_json = json.dumps(queue_dict) Function(stuff, “ExampleFn”, environment={“queues”:queue_json} ) This “just works” to have the CDK build a JSON AST that tells CloudFormation to set the ENV of your Lambda to include the name:arn pairs as JSON. With the arns of the created queues. That kind of cross referencing without using a framework is really hard; and not possible without the gross JSON since only CFn knows the stack arns until the stack has been deployed.