4 ms·

Having a state file isn't a bad idea. State files are a logical map from your code to the AWS resources. You can actually import a resource that Terraform never

Having a state file isn't a bad idea. State files are a logical map from your code to the AWS resources. You can actually import a resource that Terraform never created into a state file so that Terraform can manage it. But of course, you could just write the code to explicitly include that pre-existing resource, rather than have to run a command to tell a state file to explicitly include it. The problem with Terraform isn't that it keeps its own state file. The problem is that Terraform is just really dumb, regardless of the state file. It doesn't know how to auto-import existing resources. It doesn't know how to overwrite existing resources. It doesn't know how to detect existing resources and incorporate them into its plan. It doesn't know how to delete resources that were unexpected. It's so stupid that it basically just gives up any time any complication happens. Puppet and Ansible aren't that stupid. They both will make a best effort to deal with existing resources and work around problems. It's not the presence or absence of an external state database that make those tools "more advanced", it's simple logic.

NovemberWhiskey 4y ago
You might think that's useful behavior; personally I absolutely prefer the Terraform model of having to explicitly import things into state that weren't created by Terraform.
- throwaway892238 4y ago
  Because you get paid more per hour?
  - NovemberWhiskey 4y ago
    Why so snarky? No; because if the goal is repeatable infrastructure, I don't want the bits that are non-repeatable to be silently assumed - you just end up with a cloud-centric version of "it works on my machine".
    - nathants 4y ago
      one still must test on multiple machines before being confident it doesn’t just work on one’s machine. cloud is no different. we must test.
inkyoto 4y ago
> It doesn't know how to auto-import existing resources […] It doesn't know how to detect existing resources and incorporate them into its plan. You are offerring an oversimplified view of cloud deployments. How do you propose terraform is supposed to discern between existing resources that belong to you and the ones that belong to another colleague/project? That is the first problem and is an important one as many organisations have shared accounts where multiple distinct deployments coexist. The second problem concerns the nature of deployments: in event-driven architectures, a cloud component firing an event and the event processor are decoupled from each other. For example, an S3 bucket can fire an S3 object creation event that can be funnelled into an EventBridge, however the event processor consuming the event off the EventBridge is an entirely distinct entity and has no explicit dependency on the event source (i.e. the S3 bucket in this example). S3 buckets firing events can be later replaced with somethingn else, but the event processor will remain unaware of the change and – from the deployment perspective – it does not even need to be redeployed; terraform has no way of knowing such things. terraform can't and is not supposed to know such things as its purpose is threefold: 1. Declarative approach to the resource management – express the intentions, or the separation between «what» and «how» parts. terraform code is about «what needs to be created» but not «how it needs to created». terraform providers take over and handle the «how» part [mostly] transparently. 2. Dependency graph management – dependency management is hard. 3. Dependency tracking – it is akin to make/Makefile with the terraform state persisted in a dedicated space (locally or in a S3 bucket) as opposed to make/Makefile that makes use of the local file system and Makefile targets. > It doesn't know how to overwrite existing resources. What do you mean by overwriting existing resources? terraform can easily overwrite configuration of and metadata (such as tags) for the existing resources, but a complete overwrite of an existing resource at least in AWS – that is not even possible to the best of my knowledge.
- throwaway892238 4y ago
  > How do you propose terraform is supposed to discern between existing resources that belong to you and the ones that belong to another colleague/project? All of them "belong to you" if you have IAM access to the resource. It's trivial to restrict access to resources in AWS if you shouldn't be touching something. The fact that so many orgs don't use IAM properly is not a reason to make a tool unnecessarily difficult to use. So the question isn't if it "belongs to you", the question is whether you as the user want to do something with that particular resource that you already have permissions to modify. This could be accomplished at least a half dozen ways: 1. Is there a record in the state file of Terraform creating it? No? Then it's somebody else's resource 2. Ask the user 3. Command-line options 4. The code / DSL 5. Tags 6. Naming convention But it does none of these things. It just craps out with a generic error and you're left to manually fuck around with the state halfway through it already started applying changes in production. > 1. Declarative approach to the resource management So it's declarative. Who cares? In the real world, there are resources that Terraform didn't create that you still have to deal with. There are also resources that Terraform did create, but you lost the state file, or you moved some modules, or changed the Terraform version by mistake and can't reverse your state file version, or a million other things. "Declarative" is not some magical word that means "the real world no longer applies". > 2. Dependency graph management – dependency management is hard. It's not that hard. If you already have a DAG and algorithms that can manipulate objects in it, there's only like 2 or 3 more functions you need to manipulate dependencies in a graph. This is really basic CS stuff. Dependency tracking is the same thing. > What do you mean by overwriting existing resources? I mean if there's an AWS record that Terraform wants to create and you didn't import it, Terraform fails. This is stupid, because 1) it's not even checking if the resource already exists (it has the data providers to do this), and 2) it's making me do something manually that it could do by itself with a flag or a config or DSL entry or a prompt. The UX is fucking ridiculous. It's like whoever wrote Terraform has never actually used it, or just doesn't give a shit about blowing up production, or getting anything done in a reasonable amount of time. Classic "humans exist to serve machines", "opinionated" tech hipster bullshit. Must be the same person who did Git's UX. And impossibly, people defend it, as if technology is supposed to make our lives harder.

NovemberWhiskey 4y ago

You might think that's useful behavior; personally I absolutely prefer the Terraform model of having to explicitly import things into state that weren't created by Terraform.

throwaway892238 4y ago
Because you get paid more per hour?
- NovemberWhiskey 4y ago
  Why so snarky? No; because if the goal is repeatable infrastructure, I don't want the bits that are non-repeatable to be silently assumed - you just end up with a cloud-centric version of "it works on my machine".
  - nathants 4y ago
    one still must test on multiple machines before being confident it doesn’t just work on one’s machine. cloud is no different. we must test.

inkyoto 4y ago

> It doesn't know how to auto-import existing resources […] It doesn't know how to detect existing resources and incorporate them into its plan. You are offerring an oversimplified view of cloud deployments. How do you propose terraform is supposed to discern between existing resources that belong to you and the ones that belong to another colleague/project? That is the first problem and is an important one as many organisations have shared accounts where multiple distinct deployments coexist. The second problem concerns the nature of deployments: in event-driven architectures, a cloud component firing an event and the event processor are decoupled from each other. For example, an S3 bucket can fire an S3 object creation event that can be funnelled into an EventBridge, however the event processor consuming the event off the EventBridge is an entirely distinct entity and has no explicit dependency on the event source (i.e. the S3 bucket in this example). S3 buckets firing events can be later replaced with somethingn else, but the event processor will remain unaware of the change and – from the deployment perspective – it does not even need to be redeployed; terraform has no way of knowing such things. terraform can't and is not supposed to know such things as its purpose is threefold: 1. Declarative approach to the resource management – express the intentions, or the separation between «what» and «how» parts. terraform code is about «what needs to be created» but not «how it needs to created». terraform providers take over and handle the «how» part [mostly] transparently. 2. Dependency graph management – dependency management is hard. 3. Dependency tracking – it is akin to make/Makefile with the terraform state persisted in a dedicated space (locally or in a S3 bucket) as opposed to make/Makefile that makes use of the local file system and Makefile targets. > It doesn't know how to overwrite existing resources. What do you mean by overwriting existing resources? terraform can easily overwrite configuration of and metadata (such as tags) for the existing resources, but a complete overwrite of an existing resource at least in AWS – that is not even possible to the best of my knowledge.

throwaway892238 4y ago
> How do you propose terraform is supposed to discern between existing resources that belong to you and the ones that belong to another colleague/project? All of them "belong to you" if you have IAM access to the resource. It's trivial to restrict access to resources in AWS if you shouldn't be touching something. The fact that so many orgs don't use IAM properly is not a reason to make a tool unnecessarily difficult to use. So the question isn't if it "belongs to you", the question is whether you as the user want to do something with that particular resource that you already have permissions to modify. This could be accomplished at least a half dozen ways: 1. Is there a record in the state file of Terraform creating it? No? Then it's somebody else's resource 2. Ask the user 3. Command-line options 4. The code / DSL 5. Tags 6. Naming convention But it does none of these things. It just craps out with a generic error and you're left to manually fuck around with the state halfway through it already started applying changes in production. > 1. Declarative approach to the resource management So it's declarative. Who cares? In the real world, there are resources that Terraform didn't create that you still have to deal with. There are also resources that Terraform did create, but you lost the state file, or you moved some modules, or changed the Terraform version by mistake and can't reverse your state file version, or a million other things. "Declarative" is not some magical word that means "the real world no longer applies". > 2. Dependency graph management – dependency management is hard. It's not that hard. If you already have a DAG and algorithms that can manipulate objects in it, there's only like 2 or 3 more functions you need to manipulate dependencies in a graph. This is really basic CS stuff. Dependency tracking is the same thing. > What do you mean by overwriting existing resources? I mean if there's an AWS record that Terraform wants to create and you didn't import it, Terraform fails. This is stupid, because 1) it's not even checking if the resource already exists (it has the data providers to do this), and 2) it's making me do something manually that it could do by itself with a flag or a config or DSL entry or a prompt. The UX is fucking ridiculous. It's like whoever wrote Terraform has never actually used it, or just doesn't give a shit about blowing up production, or getting anything done in a reasonable amount of time. Classic "humans exist to serve machines", "opinionated" tech hipster bullshit. Must be the same person who did Git's UX. And impossibly, people defend it, as if technology is supposed to make our lives harder.