4 ms·

Terraform already has to manage those, and yet worse - Changes to the name of objects within the state were entirely manual until 1.0. Changes to objects undern

Terraform already has to manage those, and yet worse - Changes to the name of objects within the state were entirely manual until 1.0. Changes to objects underneath terraform often break the connection. Terraform doesn't have good tooling built in to find orphan resources, which it would if it were working with stateless objects. Terraform would be much better without state. Not 10x better, but 2x.

jen20 4y ago
This is absolutely untrue: changes to the logical name were manual until 0.6, CLI driven until recently, and encoded in config as of more recently. Changes to physical resource names is a per-provider concern and is supported wherever the upstream allows it. Orphan resources only exist as a concept if you manage entire estates in a single configurations, which is simplistic to the extreme. If I use local_file to emit a certificate authority for an EKS cluster, does that mean every other file on my file system is suddenly orphaned? What if the networking team responsible for VPCs, routing and transit search for orphaned resources in my AWS account - should that include all my instances? If you think Terraform would be better without state, I’d encourage you to put your money where your mouth is and build it.
rlabrecque 4y ago
I suspect there's probably a decent compromise to be made, like some kind of "reconcile" command, that explicitly tries to find orphan resources that reference resources it knows about, and tries to rebuild state.
janosdebugs 4y ago
I think there is a misunderstanding: when I'm talking about "name" I mean the "name" stored by the cloud provider, which is typically (arbitrarily) chosen as the idempotency key by stateless implementations. When you change the name of the resource that will, obviously, break the connection, that's a big no-no when writing Terraform code.
- cheradenine_uk 4y ago
  Sure, but isn't the artcle suggesting tags rather than the name / ID ? One problem, as mentioned in a sibling comment, is if you don't have state then how do you know a resource has been deleted? One other option might be a system that compares the "desired state" from the previous version of the config. That might be an interesting approach - keeping a history of config. But that is, of course, state :) I use TF a bit. I want to like it, but I do spend inordinate amounts of time faffing around with state files to make them match reality, and I'm not even doing particularly complex stuff. I'm told other tooling (eg: Bicep for Azure) dispenses with state entirely.
  - giaour 4y ago
    Bicep and Azure Resource Manager (ARM) are able to dispense with state only because every Azure resource is required to be individually addressable and to report its state in a consistent way, and each resource is expected to support fully idempotent creation operations. The state information is still there, but it's stored on the resource itself. Bicep/ARM still has issues with resource deletion, though. The default deployment behavior is to ignore resources that aren't described in a deployment template, so if you remove a VM from your template and redeploy, the VM will keep running until you manually delete it. There are a couple ways around this issue, but they all rely on having state external to the resource itself. Disclaimer: I work on Bicep/ARM and think it's pretty great, but it's not perfect.
  - janosdebugs 4y ago
    Not every provider supports tags, or they don't support it for all resources. Sure, if you only use AWS this may be a solution, but not for everyone. Regarding the state, it took me a very long time to come up with good organization for my code, but it works once you've gotten used to it. I really only need to mess with state files when there is a bug in the provider like the aforementioned GitHub issue.
rbongers 4y ago
I don't quite follow - why would it be able to find orphaned resources if it were working with stateless objects?
- sly010 4y ago
  Presumably because it would already have all the necessary code to find/enumerate existing resources.

jen20 4y ago

This is absolutely untrue: changes to the logical name were manual until 0.6, CLI driven until recently, and encoded in config as of more recently. Changes to physical resource names is a per-provider concern and is supported wherever the upstream allows it. Orphan resources only exist as a concept if you manage entire estates in a single configurations, which is simplistic to the extreme. If I use local_file to emit a certificate authority for an EKS cluster, does that mean every other file on my file system is suddenly orphaned? What if the networking team responsible for VPCs, routing and transit search for orphaned resources in my AWS account - should that include all my instances? If you think Terraform would be better without state, I’d encourage you to put your money where your mouth is and build it.

rlabrecque 4y ago

I suspect there's probably a decent compromise to be made, like some kind of "reconcile" command, that explicitly tries to find orphan resources that reference resources it knows about, and tries to rebuild state.

janosdebugs 4y ago

I think there is a misunderstanding: when I'm talking about "name" I mean the "name" stored by the cloud provider, which is typically (arbitrarily) chosen as the idempotency key by stateless implementations. When you change the name of the resource that will, obviously, break the connection, that's a big no-no when writing Terraform code.

cheradenine_uk 4y ago
Sure, but isn't the artcle suggesting tags rather than the name / ID ? One problem, as mentioned in a sibling comment, is if you don't have state then how do you know a resource has been deleted? One other option might be a system that compares the "desired state" from the previous version of the config. That might be an interesting approach - keeping a history of config. But that is, of course, state :) I use TF a bit. I want to like it, but I do spend inordinate amounts of time faffing around with state files to make them match reality, and I'm not even doing particularly complex stuff. I'm told other tooling (eg: Bicep for Azure) dispenses with state entirely.
- giaour 4y ago
  Bicep and Azure Resource Manager (ARM) are able to dispense with state only because every Azure resource is required to be individually addressable and to report its state in a consistent way, and each resource is expected to support fully idempotent creation operations. The state information is still there, but it's stored on the resource itself. Bicep/ARM still has issues with resource deletion, though. The default deployment behavior is to ignore resources that aren't described in a deployment template, so if you remove a VM from your template and redeploy, the VM will keep running until you manually delete it. There are a couple ways around this issue, but they all rely on having state external to the resource itself. Disclaimer: I work on Bicep/ARM and think it's pretty great, but it's not perfect.
- janosdebugs 4y ago
  Not every provider supports tags, or they don't support it for all resources. Sure, if you only use AWS this may be a solution, but not for everyone. Regarding the state, it took me a very long time to come up with good organization for my code, but it works once you've gotten used to it. I really only need to mess with state files when there is a bug in the provider like the aforementioned GitHub issue.

rbongers 4y ago

I don't quite follow - why would it be able to find orphaned resources if it were working with stateless objects?

sly010 4y ago
Presumably because it would already have all the necessary code to find/enumerate existing resources.