5 ms·

If you wanted to store the state in Git, you would probably want to at least encrypt secrets. This is not something that Terraform supports now (theory is that

If you wanted to store the state in Git, you would probably want to at least encrypt secrets. This is not something that Terraform supports now (theory is that this is to push their commercial offering) but it’s one of the main feature requests for OpenTofu. With this in place, and ideally with allowing folks to define custom backends using the plugin interface (decoupled from core, versioned) this story could make a ton of sense. Disclosure: I’m on the steering committee of OpenTofu.

leetrout 3y ago
Its the biggest gap in TF in my opinion. The fact the Vault provider happily slurps all your secrets into state vs using hashes or similar blows my mind. I am convinced HC just is out of touch at this point because "treat the state as sensitive" is not enough. There are many cases I would accept simple hashes vs client side encryption, too. Everyone is going to have different needs but it makes no sense HC ignored making this better for so long.
- fishnchips 3y ago
  I agree that the lack of native state and/or secret encryption is a serious limitation. In Terraform’s defense, it’s often the fault of providers and/or APIs.
  - jacurtis 3y ago
    An easy solution would be to just encrypt the whole statefile. This would work the same as state locking works now. You can apply an extra provider for state encryption/decryption just like you do for state locking/unlocking. It's already been requested (with pull requests) for Terraform for a while now and Hashicorp keeps rejecting it, presumably because it would undermine features of Terraform enterprise. OpenTofu is already considering implementing this feature as a result.
Longwelwind 3y ago
Why would you want to store the state in Git? This means that if you have a CD pipeline (triggered itself by a commit) that does `terraform deploy`, you'd need to have your CD pipeline do commit _again_ to save the changes to the TF state files. On top of that, you lose the ability to re-run a previous CD pipeline to rollback to a previous version of your infrastructure. The goal of Git is just to store the lifecycle of a piece of software, imo. It's not the reponsibility of the repo to know the state of the deployment.
- fishnchips 3y ago
  You’d just use Git as fancy storage with some extra features. You could use a separate repo for that. But I agree that the gain from that is not crazy.
tremon 3y ago
Isn't that a misuse of Git? I would consider Terraform state files to be artifacts, which means they have no place being tracked in a source control system. The natural way to use git to manage incremental deployments would be to use git history: given the tag/commit of the previous deployment, find the diff between the resources, and only deploy that diff. Or, put differently: the "previous state" is already recorded in git, under the earlier tag/commit, as the article notes.
- fishnchips 3y ago
  Not sure if it’s misuse. Versioning your state is not a terrible idea in general. Using Git for that is kinda natural for reasonably sized states. I personally would store the state in a different repo than the infra definitions, though.
  - tremon 3y ago
    How would you handle branching and merging of state files? Is there a valid reason to have different state files on different branches (our use case: different branches represent different DTAP phases, and hence deploy to different environments)? What would merge conflict resolution look like for these state files? What would happen if I deploy an environment from one branch that had previously been deployed from a different branch? If the answer is "you should never branch and merge state files", then the corollary is "you should never put state files in source control".
  - jacurtis 3y ago
    This isn't versioning your state though. This is using git commit history to essentially determine state at "apply-time". Versioning your state is a good idea and is already easy to do. For example if you use the S3 backend you can enable object versioning on your bucket and then everytime the state is updated it is versioning and you can rollback state that way. It is a good practice. Similarly, you could use the local state backend, which just keeps state in a json file in your project and then you could commit that json file to version control. Now git is versioning your state. But the author is not suggesting this, they are suggesting to not have a state file at all. You look at previous commits to determine state. So for example if you create a new resource for an ec2 instance in your code and run terraform apply, how does Terraform know it is new and needs to create this ec2 instance as opposed to updating an existing one? Normally it would look at the statefile and if it isn't in there, then tf knows to create it, if it is already in state then it knows to update it and reconcile the metadata in state to the declarative state in the code. But without a statefile this can't be done. So the author looks in the git history at the previous commit. If you can find the resource block in the commit history you know what it was last set to. If it doesn't exist in the history but exists in the current file, then it is presumed that the resource is new. If the resource was in the previous history but is different in the current commit then it is presumed that those changes need to be applied. I'm sure this works in simple use-cases. But at any moderate scale this could break down very quickly.

leetrout 3y ago

Its the biggest gap in TF in my opinion. The fact the Vault provider happily slurps all your secrets into state vs using hashes or similar blows my mind. I am convinced HC just is out of touch at this point because "treat the state as sensitive" is not enough. There are many cases I would accept simple hashes vs client side encryption, too. Everyone is going to have different needs but it makes no sense HC ignored making this better for so long.

fishnchips 3y ago
I agree that the lack of native state and/or secret encryption is a serious limitation. In Terraform’s defense, it’s often the fault of providers and/or APIs.
- jacurtis 3y ago
  An easy solution would be to just encrypt the whole statefile. This would work the same as state locking works now. You can apply an extra provider for state encryption/decryption just like you do for state locking/unlocking. It's already been requested (with pull requests) for Terraform for a while now and Hashicorp keeps rejecting it, presumably because it would undermine features of Terraform enterprise. OpenTofu is already considering implementing this feature as a result.

Longwelwind 3y ago

Why would you want to store the state in Git? This means that if you have a CD pipeline (triggered itself by a commit) that does `terraform deploy`, you'd need to have your CD pipeline do commit _again_ to save the changes to the TF state files. On top of that, you lose the ability to re-run a previous CD pipeline to rollback to a previous version of your infrastructure. The goal of Git is just to store the lifecycle of a piece of software, imo. It's not the reponsibility of the repo to know the state of the deployment.

fishnchips 3y ago
You’d just use Git as fancy storage with some extra features. You could use a separate repo for that. But I agree that the gain from that is not crazy.

tremon 3y ago

Isn't that a misuse of Git? I would consider Terraform state files to be artifacts, which means they have no place being tracked in a source control system. The natural way to use git to manage incremental deployments would be to use git history: given the tag/commit of the previous deployment, find the diff between the resources, and only deploy that diff. Or, put differently: the "previous state" is already recorded in git, under the earlier tag/commit, as the article notes.

fishnchips 3y ago
Not sure if it’s misuse. Versioning your state is not a terrible idea in general. Using Git for that is kinda natural for reasonably sized states. I personally would store the state in a different repo than the infra definitions, though.
- tremon 3y ago
  How would you handle branching and merging of state files? Is there a valid reason to have different state files on different branches (our use case: different branches represent different DTAP phases, and hence deploy to different environments)? What would merge conflict resolution look like for these state files? What would happen if I deploy an environment from one branch that had previously been deployed from a different branch? If the answer is "you should never branch and merge state files", then the corollary is "you should never put state files in source control".
- jacurtis 3y ago
  This isn't versioning your state though. This is using git commit history to essentially determine state at "apply-time". Versioning your state is a good idea and is already easy to do. For example if you use the S3 backend you can enable object versioning on your bucket and then everytime the state is updated it is versioning and you can rollback state that way. It is a good practice. Similarly, you could use the local state backend, which just keeps state in a json file in your project and then you could commit that json file to version control. Now git is versioning your state. But the author is not suggesting this, they are suggesting to not have a state file at all. You look at previous commits to determine state. So for example if you create a new resource for an ec2 instance in your code and run terraform apply, how does Terraform know it is new and needs to create this ec2 instance as opposed to updating an existing one? Normally it would look at the statefile and if it isn't in there, then tf knows to create it, if it is already in state then it knows to update it and reconcile the metadata in state to the declarative state in the code. But without a statefile this can't be done. So the author looks in the git history at the previous commit. If you can find the resource block in the commit history you know what it was last set to. If it doesn't exist in the history but exists in the current file, then it is presumed that the resource is new. If the resource was in the previous history but is different in the current commit then it is presumed that those changes need to be applied. I'm sure this works in simple use-cases. But at any moderate scale this could break down very quickly.