3 ms·
Because the source might no longer exist. Same as the current state might no longer exist. You can't diff something you don't know about and thus cannot create
by oneplane 3y ago
Because the source might no longer exist. Same as the current state might no longer exist. You can't diff something you don't know about and thus cannot create a transaction to deal with it.
Three examples, but first context:
You create some .tf files that create a bucket on s3 on AWS, and then create an object inside the bucket. This creates a graph data structure that describes the AWS provider, the S3 bucket and the S3 Object. You then apply this desired state, which causes terraform to create some resources in AWS and then describe the state as it was last known in the state file.
Scenario 1: You delete the object from AWS but don't tell Terraform about it. You re-run apply, terraform notices that it is gone and re-creates it, so far so good. Even Ansible and SaltStack could do this.
Scenario 2: You delete the object from the desired state in your .tf files. You re-run apply and terraform notices that you used to have an S3 Object in your desired state (because your state file says so), but since it is no longer there it goes to talk to the AWS API to see if the object is still there and if it is, it will delete it (or, plans to delete it and then it is up to you to apply or discard that plan). This is not possible if you don't record your previous intent. One-way tools fail here.
Scenario 3: you make a mess where you remove the object from your .tf files and also manually edit the state file and remove the object there as well. Terraform now knows nothing about what the world used to look like. But you then decide that the S3 bucket can be removed and since it's all managed resources anyway (no dirty clickOps tricks in the console) you tell Terraform to destroy the bucket. Terraform doesn't see any dependencies, and asks aws to nuke the bucket. But since the bucket isn't empty, you get an error. Terraform doesn't know anything about any objects in the bucket, and thus it couldn't clean it up.
Now, the scenarios here are oversimplified, and you would probably not setup an IaC tool to manage 1 bucket and 1 object if that is all you'll ever configure. But when you start creating multiple environments that each need their own configuration drift control and controlled resource destruction you really can't function without it.
- gaganyaan 3y agoIs there a functional difference between that approach and using tags on existing resources? I know Terraform doesn't use tags for this because not everything supports them, but in K8s, scenario 2 is handled by just saying "make everything using this tag look exactly like this". If there's extra resources not in the YAML with that tag, they'll get cleaned up. In general, I like that approach better, because it just uses the state that already exists. It would be kind of nice if Terraform had a mode where it avoided creating extra state where possible.
- oneplane 3y agoYes, because tags do not express where in a graph a node might be. It also gives you a new problem, because to find out what the previously known state was, you now have to read all tags on all resources. In Kubernetes, the state reconciliation happens by controllers based on information in etcd, in a way, etcd is what the statefile in terraform is. The apiserver is what the 'current' state is, including metadata and event fields, and any on-disk manifests you might have would be the desired state. The only big difference between the Kubernetes reconciler and terraform is that event-driven nature. Terraform tends to be a series of one-shots where Kubernetes (the controller manager and controllers) is a constant stream of reconciliation events. The overarching theme in both is 'desired state reconciliation via declarative configuration'. As for client-side apply and server-side apply in K8S, you're essentially having the same deal as terraform: you feed it a manifest of what you want, and it will figure out if it is new, old, needs to be updated or needs to be destroyed.
- ta1243 3y agoThe concept I struggle with is "the desired state, the last known state and the current state" Why do you need to know the last known state. You have the desired state and the current state, you run your code and you reconcile the current state to the desired state I get why this is impossible/expensive to get the current state with AWS, which doesn't expose a simple "show state" API, but managing a state store is extra work and thus extra fragility. However ideally I'd like to list credentials and providers to manage, then say "manage all resources with some form of tag "managedByTerraform=12345" (this could even be in the resource name -- for my fortigate management my scripts manage firewall rules, addresses, etc which start with AUTO_, and ignore the other objects) It would then run, generate a lock resource, find the current state, compare with desired state, create/delete elements to reconcile actuality with desire, implement, free the lock Then all I need to do is have that human readable file and I can get back to where I am If I lose the state, or it gets corrupt, I'm in for a world of hurt with terraform.
- oneplane 3y agoWhat you propose is computationally rather expensive, and on an API level really hard to implement uniformly. And thus the state was born. Trying to do it without state has been tried plenty of times, and it's bad. (Chef, SaltStack, Ansible, CFengine, Puppet to name a few) It only works somewhat okay if you're simply writing to a single managed unit, i.e. a file. You know what all the contents of the file have to be, so you can replace it wholesale. This does not work for a graph of resources that interact with each other. Separately, this also doesn't work with a shared responsibility model and doesn't work with deleted code. Terraform has a state because that is required to solve the problem in a reasonable manner. It doesn't have state just because it was thought to be a fun exercise ;-) If the problem being solved is not _your_ problem, it is highly likely that using terraform at all is also not going to help you. > If I lose the state, or it gets corrupt, I'm in for a world of hurt with terraform. And that's why you secure and version your state. I.e. using S3 and versioning and a restrictive policy.