5 ms·

Genuine question for DevOps people: Other than the fact it seems to be an industry standard so it's good for your job prospects, what are the benefits to Terra

by rcarr 3y ago

Genuine question for DevOps people: Other than the fact it seems to be an industry standard so it's good for your job prospects, what are the benefits to Terraform over CloudFormation/CDK or whatever the equivalent is for your particular cloud provider? Most companies/people pick a provider and then stick with it and it doesn't seem like there's much portability between configurations if you do decide to switch providers later down the line so I'm not sure what the benefits are. I haven't delved into Terraform yet but I tried doing a project in Pulumi once and felt by the end of it that I might as well have just wrote it in AWS CDK directly.

jahsome 3y ago
Third-party integrations and the universality/reusability across multiple products and familiarity of HCL are big for me.
x3n0ph3n3 3y ago
I have used both terraform and cloudformation substantially and they each have pros and cons. One thing terraform has over cloudformation is its rapid support for new services and features. AWS has done an awful job ensuring that cloudformation support is part of each team's definition of "done" for each release. It just doesn't get the support it really needs from AWS.
- koolba 3y ago
  CloudFormation is the ugly step child of AWS. It has bugs that have languished for years
  - viraptor 3y ago
    Or weird missing functionality like some resources being impossible to tag from the stacks, or new services/options lagging months behind when TF implemented support for them.
Centigonal 3y ago
I worked closely with the folks that wrote our platform's IaC, first in CDK, then in Terraform. I wrote a bit of CDK and zero TF myself, but here are some of the reasons we switched: A big plus is that Terraform works outside of AWS land. CDK is a nightmare to work with. You're writing with programming-language syntax, which tempts you to write dynamic stuff - but everything still compiles down to declarative CFN, which just makes the ergonomics feel limited. The L2 and L3 constructs have a lot of implicit defaults that came back to bite us later. With CDK you get synth and deploy, which felt like a black box. Minor changes would do the same 8 minute long deploy process as large infrastructure refactors. Switching to TF significantly sped up our builds for minor commits. There might be a better way to do this with CDK (maybe deploying separate apps for each part of our infrastructure) and we may have just missed it.
thedougd 3y ago
Providers I regularly use, even mixed in a single project. There are others I could use if they were available. AWS GitHub Opsgenie Okta Scalr TLS DNS
- koolba 3y ago
  You forgot the greatest escape hatch of all: null
yellowapple 3y ago
- In the event that you are working with different cloud providers, Terraform is one thing to learn that then applies to all of them, as opposed to learning each provider's bespoke infra-as-code offering. Most companies stick to one PaaS/IaaS, but individual personnel ain't necessarily as limited over the courses of their careers. - Not all cloud providers have an infra-as-code offering of their own in the first place (especially true with traditional server hosts), whereas pretty much every provider with some sort of API most likely has a Terraform provider implemented for it. - Terraform providers include more than just PaaS/IaaS providers / server hosts; for example, my current job includes provisioning Datadog metrics and PagerDuty alerts alongside applications' AWS infra in the same per-app Terraform codebase, and a previous job entailed configuring Keycloak instances/realms/roles/etc. via Terraform.
- androidbishop 3y ago
  Also pretty neat that there's a Terraform provider for Kubernetes native resources.
  - mcfedr 3y ago
    Personally it's the only way to work with kubernetes. Terraforms change control is excellent. Cannot go back to helms upgrade and how it does what you want.
dgrin91 3y ago
Companies choose providers and tend to stick with them, but people don't always stick with companies. If I know TF there is a decent chance my skills will be applicable when I change companies. Also some big corps run their own internal datacenters and have cloud-like interfaces with them. You can write TF providers for that (its not going to be as nice as the public cloud ones, but still nice). Then you can utilize Terraforms multi-provider functionality to have 1 project manage deployments on multiple clouds that include on-prem. Also terraforms multi-provider functionality is also useful for non aws/azure/gcp such as Cloudflare. As far as I know CDK does not support that.
RulerOf 3y ago
I've got a lot of opinions here, but the only one I'll share is that HCL knocks the socks off of json and yaml. Json is too rigid. YAML is too nested. HCL gets this just right. Venturing away from opinions, the provider ecosystem with terraform enables some wonderful design options. For example, I have a module template that takes some basic container configs (e.g. ports, healthchecks) and a GitHub URL, then stands the service up on ECS and configures CI in the linked repo. CF can't do that.
abrookewood 3y ago
After you have waited 20 minutes for CloudFormation to fail and tell you that it can't delete a resource (but won't tell you why), and this is the third time it has happened in a week, you start looking at alternatives.
- androidbishop 3y ago
  This 1000%. Also recently discovered Google Deployment Manager is shit for the exact same reasons. I honestly don't get it.
333throwaway342 3y ago
> Most companies/people pick a provider and then stick with it and it doesn't seem like there's much portability between configurations if you do decide to switch providers later down the line so I'm not sure what the benefits are. This smells like kubernetes > Terraform over CloudFormation/CDK They both work. It's more about which providers you need.
Illotus 3y ago
> Other than the fact it seems to be an industry standard so it's good for your job prospects, what are the benefits to Terraform over CloudFormation/CDK or whatever the equivalent is for your particular cloud provider? For me the killer feature is that both plan and apply show the actual diff of changes vs running infrastructure. It makes understanding effects of changes much easier.
- Bellyache5 3y ago
  Agreed, Terraform does a good job of this. But CloudFormation & CDK can also do this via Change Sets and CDK diff. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-changesets.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui... https://blog.mikaeels.com/what-does-the-aws-cdk-diff-command-do https://blog.mikaeels.com/what-does-the-aws-cdk-diff-command...
nuker 3y ago
Im 10 years working with AWS. I strongly prefer Cloudformation, just separate things smartly between stacks. It has export/import for stack outputs too. Just look at the “root module” mess in this discussion and you’ll get why.
androidbishop 3y ago
Terraform, and by extension HCL, is more powerful and flexible. It can be used across clouds. It has providers for all kinds of things, like kubernetes. It can be abstracted and modularized. It supports cool features like workspaces and junk, depending on how you want to use it. Also recently I was forced to use Google Cloud Deployment Manager scripts for some legacy project we were migrating to Terraform, and I was shocked at how buggy and useless it was. Failed to create resources for no discernible reason, couldn't update existing resources with dependencies, couldn't delete resources, was just unfathomably shit all around. Finished the Terraform migration earlier this morning and everything went off without a hitch, plus we got more coverage for stuff Deployment Manager doesn't support. It's also organized much nicer now, with versioned modules and what-have-you. Cloudformation is ugly and again, surprisingly isn't well supported by AWS. I don't understand how it's possible, but terraform providers seem to be more up to date with products and APIs. Maybe that's just me but I've seen others complain about the same thing.
- wodenokoto 3y ago
  Isn’t google cloud deployment just bash calls to the google cloud cli disguised as declarations by way of yaml?
  - androidbishop 3y ago
    Considering the fact that you can inject python code blocks, I kind of doubt it. It also makes API calls that populates a dashboard with an inventory of resources created, so it seems to be more of an api wrapper like other IaC solutions.
cwp 3y ago
It's subtle and so difficult to see the differences at smaller scales. If you're going to provision a handful of EC2 instances, all the tools work fine. I think HCL is an under appreciated aspect of Terraform. It was kinda awful for a while, but it's gotten a lot better and much easier to work with. It hits a sweet spot between data languages like JSON and YAML and fully-general programming languages like Python. Take CloudFormation. The "native" language is JSON, and they've added YAML support for better ergonomics. But JSON is just not expressive enough. You end up with "pseudoparameters" and "function calls" layered on top. Attribute names doubling as type declarations, deeply nested layers of structure and incredible amounts of repetitious complexity just to be able to express all the details need to handle even moderate amounts of infrastructure. So, ok, AWS recognizes this and they provide CDK so you can wring out all the repetion using a real programming language - pick your favourite one, a bunch are supported. That helps some, but now you've got the worst of both worlds. It's not "just JSON" anymore. You need a full programming environment. The CDK, let's say the Python version, has to run on the right interpreter. It has a lot of package dependencies, and you'll probably want to run it in a virtualenv, or maybe a container. And it's got the full power of Python, so you might have sources of non-determinism that give you subtle errors and bugs. Maybe it's daylight saving gotchas or hidden dependencies on data that it pulls in from the net. This can sound paranoid, but these things do start to bite if you have enough scale and enough time. And then, all that Python code is just a front end to the JSON, so you get some insulation from it, but sometimes you're going to have to reason about the JSON it's producing. HCL, despite its warts, avoids the problems with these extremes. It's enough of a programming language that you can just use named, typed variables to deal with configuration, instead of all the { "Fn::GetAtt" : ["ObjectName", "AttName"] } nonsense that CloudFormation will put you through. And the ability to create modules that can call each other is sooo important for wringing out all the repetition that these configurations seem to generate. On the other hand, it's not fully general, so you don't have to deal with things like loops, recursion, and so on. This lack of power in the language enables more power in the tools. Things like the plan/apply distinction, automatically tracking dependencies between resources, targeting specific resources, move blocks etc. would be difficult or impossible with a language as powerful as Python. HCL isn't the only language in this space - see CUE and Dhall, for example - but it's undoubtedly the most widely used. And it makes a real difference in practice.
solatic 3y ago
> your particular cloud provider Just this week, I wrote a Terraform module that uses the GCP, Kubernetes, and Cloudflare providers to allow us to bring up a single business-need (that will be needed, hopefully, many times in the future) that spans those three layers of the stack. 200 lines of Terraform written in an afternoon replaced a janky 2,000 line over-engineered Python microservice (including much retry and failure-handling logic that Terraform gives you for free) whose original author (not DevOps) moved on to better pastures. CDK is fine if you're all-in on AWS. It has its tradeoffs compared to Terraform. Pick the right tool for the job.
maccard 3y ago
We don't just have AWS resources. Our CI pipelines are managed by terraform [0], they communicate with GitHub [1]. I like that it's declarative and limited, it stops people trying to do "clever shit" with our infra, which is complicated enough as it is. [0] https://buildkite.com/blog/manage-your-ci-cd-resources-as-code-with-terraform https://buildkite.com/blog/manage-your-ci-cd-resources-as-co... [1] https://registry.terraform.io/providers/integrations/github/latest/docs https://registry.terraform.io/providers/integrations/github/...
raffraffraff 3y ago
For me personally, I chose terraform because it can work with AWS and a heap of other 3rd party services and software (Cloudflare, PostgreSQL, Keycloak, Kubernetes/Helm, Github, Azure)

by rcarr 3y ago

jahsome 3y ago

Third-party integrations and the universality/reusability across multiple products and familiarity of HCL are big for me.

x3n0ph3n3 3y ago

I have used both terraform and cloudformation substantially and they each have pros and cons. One thing terraform has over cloudformation is its rapid support for new services and features. AWS has done an awful job ensuring that cloudformation support is part of each team's definition of "done" for each release. It just doesn't get the support it really needs from AWS.

koolba 3y ago
CloudFormation is the ugly step child of AWS. It has bugs that have languished for years
- viraptor 3y ago
  Or weird missing functionality like some resources being impossible to tag from the stacks, or new services/options lagging months behind when TF implemented support for them.

Centigonal 3y ago

I worked closely with the folks that wrote our platform's IaC, first in CDK, then in Terraform. I wrote a bit of CDK and zero TF myself, but here are some of the reasons we switched: A big plus is that Terraform works outside of AWS land. CDK is a nightmare to work with. You're writing with programming-language syntax, which tempts you to write dynamic stuff - but everything still compiles down to declarative CFN, which just makes the ergonomics feel limited. The L2 and L3 constructs have a lot of implicit defaults that came back to bite us later. With CDK you get synth and deploy, which felt like a black box. Minor changes would do the same 8 minute long deploy process as large infrastructure refactors. Switching to TF significantly sped up our builds for minor commits. There might be a better way to do this with CDK (maybe deploying separate apps for each part of our infrastructure) and we may have just missed it.

thedougd 3y ago

Providers I regularly use, even mixed in a single project. There are others I could use if they were available. AWS GitHub Opsgenie Okta Scalr TLS DNS

koolba 3y ago
You forgot the greatest escape hatch of all: null

yellowapple 3y ago

- In the event that you are working with different cloud providers, Terraform is one thing to learn that then applies to all of them, as opposed to learning each provider's bespoke infra-as-code offering. Most companies stick to one PaaS/IaaS, but individual personnel ain't necessarily as limited over the courses of their careers. - Not all cloud providers have an infra-as-code offering of their own in the first place (especially true with traditional server hosts), whereas pretty much every provider with some sort of API most likely has a Terraform provider implemented for it. - Terraform providers include more than just PaaS/IaaS providers / server hosts; for example, my current job includes provisioning Datadog metrics and PagerDuty alerts alongside applications' AWS infra in the same per-app Terraform codebase, and a previous job entailed configuring Keycloak instances/realms/roles/etc. via Terraform.

androidbishop 3y ago
Also pretty neat that there's a Terraform provider for Kubernetes native resources.
- mcfedr 3y ago
  Personally it's the only way to work with kubernetes. Terraforms change control is excellent. Cannot go back to helms upgrade and how it does what you want.

dgrin91 3y ago

Companies choose providers and tend to stick with them, but people don't always stick with companies. If I know TF there is a decent chance my skills will be applicable when I change companies. Also some big corps run their own internal datacenters and have cloud-like interfaces with them. You can write TF providers for that (its not going to be as nice as the public cloud ones, but still nice). Then you can utilize Terraforms multi-provider functionality to have 1 project manage deployments on multiple clouds that include on-prem. Also terraforms multi-provider functionality is also useful for non aws/azure/gcp such as Cloudflare. As far as I know CDK does not support that.

RulerOf 3y ago

I've got a lot of opinions here, but the only one I'll share is that HCL knocks the socks off of json and yaml. Json is too rigid. YAML is too nested. HCL gets this just right. Venturing away from opinions, the provider ecosystem with terraform enables some wonderful design options. For example, I have a module template that takes some basic container configs (e.g. ports, healthchecks) and a GitHub URL, then stands the service up on ECS and configures CI in the linked repo. CF can't do that.

abrookewood 3y ago

After you have waited 20 minutes for CloudFormation to fail and tell you that it can't delete a resource (but won't tell you why), and this is the third time it has happened in a week, you start looking at alternatives.

androidbishop 3y ago
This 1000%. Also recently discovered Google Deployment Manager is shit for the exact same reasons. I honestly don't get it.

333throwaway342 3y ago

> Most companies/people pick a provider and then stick with it and it doesn't seem like there's much portability between configurations if you do decide to switch providers later down the line so I'm not sure what the benefits are. This smells like kubernetes > Terraform over CloudFormation/CDK They both work. It's more about which providers you need.

Illotus 3y ago

> Other than the fact it seems to be an industry standard so it's good for your job prospects, what are the benefits to Terraform over CloudFormation/CDK or whatever the equivalent is for your particular cloud provider? For me the killer feature is that both plan and apply show the actual diff of changes vs running infrastructure. It makes understanding effects of changes much easier.

Bellyache5 3y ago
Agreed, Terraform does a good job of this. But CloudFormation & CDK can also do this via Change Sets and CDK diff. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-changesets.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui... https://blog.mikaeels.com/what-does-the-aws-cdk-diff-command-do https://blog.mikaeels.com/what-does-the-aws-cdk-diff-command...

nuker 3y ago

Im 10 years working with AWS. I strongly prefer Cloudformation, just separate things smartly between stacks. It has export/import for stack outputs too. Just look at the “root module” mess in this discussion and you’ll get why.

androidbishop 3y ago

Terraform, and by extension HCL, is more powerful and flexible. It can be used across clouds. It has providers for all kinds of things, like kubernetes. It can be abstracted and modularized. It supports cool features like workspaces and junk, depending on how you want to use it. Also recently I was forced to use Google Cloud Deployment Manager scripts for some legacy project we were migrating to Terraform, and I was shocked at how buggy and useless it was. Failed to create resources for no discernible reason, couldn't update existing resources with dependencies, couldn't delete resources, was just unfathomably shit all around. Finished the Terraform migration earlier this morning and everything went off without a hitch, plus we got more coverage for stuff Deployment Manager doesn't support. It's also organized much nicer now, with versioned modules and what-have-you. Cloudformation is ugly and again, surprisingly isn't well supported by AWS. I don't understand how it's possible, but terraform providers seem to be more up to date with products and APIs. Maybe that's just me but I've seen others complain about the same thing.

wodenokoto 3y ago
Isn’t google cloud deployment just bash calls to the google cloud cli disguised as declarations by way of yaml?
- androidbishop 3y ago
  Considering the fact that you can inject python code blocks, I kind of doubt it. It also makes API calls that populates a dashboard with an inventory of resources created, so it seems to be more of an api wrapper like other IaC solutions.

cwp 3y ago

It's subtle and so difficult to see the differences at smaller scales. If you're going to provision a handful of EC2 instances, all the tools work fine. I think HCL is an under appreciated aspect of Terraform. It was kinda awful for a while, but it's gotten a lot better and much easier to work with. It hits a sweet spot between data languages like JSON and YAML and fully-general programming languages like Python. Take CloudFormation. The "native" language is JSON, and they've added YAML support for better ergonomics. But JSON is just not expressive enough. You end up with "pseudoparameters" and "function calls" layered on top. Attribute names doubling as type declarations, deeply nested layers of structure and incredible amounts of repetitious complexity just to be able to express all the details need to handle even moderate amounts of infrastructure. So, ok, AWS recognizes this and they provide CDK so you can wring out all the repetion using a real programming language - pick your favourite one, a bunch are supported. That helps some, but now you've got the worst of both worlds. It's not "just JSON" anymore. You need a full programming environment. The CDK, let's say the Python version, has to run on the right interpreter. It has a lot of package dependencies, and you'll probably want to run it in a virtualenv, or maybe a container. And it's got the full power of Python, so you might have sources of non-determinism that give you subtle errors and bugs. Maybe it's daylight saving gotchas or hidden dependencies on data that it pulls in from the net. This can sound paranoid, but these things do start to bite if you have enough scale and enough time. And then, all that Python code is just a front end to the JSON, so you get some insulation from it, but sometimes you're going to have to reason about the JSON it's producing. HCL, despite its warts, avoids the problems with these extremes. It's enough of a programming language that you can just use named, typed variables to deal with configuration, instead of all the { "Fn::GetAtt" : ["ObjectName", "AttName"] } nonsense that CloudFormation will put you through. And the ability to create modules that can call each other is sooo important for wringing out all the repetition that these configurations seem to generate. On the other hand, it's not fully general, so you don't have to deal with things like loops, recursion, and so on. This lack of power in the language enables more power in the tools. Things like the plan/apply distinction, automatically tracking dependencies between resources, targeting specific resources, move blocks etc. would be difficult or impossible with a language as powerful as Python. HCL isn't the only language in this space - see CUE and Dhall, for example - but it's undoubtedly the most widely used. And it makes a real difference in practice.

solatic 3y ago

> your particular cloud provider Just this week, I wrote a Terraform module that uses the GCP, Kubernetes, and Cloudflare providers to allow us to bring up a single business-need (that will be needed, hopefully, many times in the future) that spans those three layers of the stack. 200 lines of Terraform written in an afternoon replaced a janky 2,000 line over-engineered Python microservice (including much retry and failure-handling logic that Terraform gives you for free) whose original author (not DevOps) moved on to better pastures. CDK is fine if you're all-in on AWS. It has its tradeoffs compared to Terraform. Pick the right tool for the job.

maccard 3y ago

We don't just have AWS resources. Our CI pipelines are managed by terraform [0], they communicate with GitHub [1]. I like that it's declarative and limited, it stops people trying to do "clever shit" with our infra, which is complicated enough as it is. [0] https://buildkite.com/blog/manage-your-ci-cd-resources-as-code-with-terraform https://buildkite.com/blog/manage-your-ci-cd-resources-as-co... [1] https://registry.terraform.io/providers/integrations/github/latest/docs https://registry.terraform.io/providers/integrations/github/...

raffraffraff 3y ago

For me personally, I chose terraform because it can work with AWS and a heap of other 3rd party services and software (Cloudflare, PostgreSQL, Keycloak, Kubernetes/Helm, Github, Azure)