3 ms·

Isn't that a misuse of Git? I would consider Terraform state files to be artifacts, which means they have no place being tracked in a source control system. The

by tremon 3y ago

Isn't that a misuse of Git? I would consider Terraform state files to be artifacts, which means they have no place being tracked in a source control system. The natural way to use git to manage incremental deployments would be to use git history: given the tag/commit of the previous deployment, find the diff between the resources, and only deploy that diff. Or, put differently: the "previous state" is already recorded in git, under the earlier tag/commit, as the article notes.

fishnchips 3y ago
Not sure if it’s misuse. Versioning your state is not a terrible idea in general. Using Git for that is kinda natural for reasonably sized states. I personally would store the state in a different repo than the infra definitions, though.
- tremon 3y ago
  How would you handle branching and merging of state files? Is there a valid reason to have different state files on different branches (our use case: different branches represent different DTAP phases, and hence deploy to different environments)? What would merge conflict resolution look like for these state files? What would happen if I deploy an environment from one branch that had previously been deployed from a different branch? If the answer is "you should never branch and merge state files", then the corollary is "you should never put state files in source control".
- jacurtis 3y ago
  This isn't versioning your state though. This is using git commit history to essentially determine state at "apply-time". Versioning your state is a good idea and is already easy to do. For example if you use the S3 backend you can enable object versioning on your bucket and then everytime the state is updated it is versioning and you can rollback state that way. It is a good practice. Similarly, you could use the local state backend, which just keeps state in a json file in your project and then you could commit that json file to version control. Now git is versioning your state. But the author is not suggesting this, they are suggesting to not have a state file at all. You look at previous commits to determine state. So for example if you create a new resource for an ec2 instance in your code and run terraform apply, how does Terraform know it is new and needs to create this ec2 instance as opposed to updating an existing one? Normally it would look at the statefile and if it isn't in there, then tf knows to create it, if it is already in state then it knows to update it and reconcile the metadata in state to the declarative state in the code. But without a statefile this can't be done. So the author looks in the git history at the previous commit. If you can find the resource block in the commit history you know what it was last set to. If it doesn't exist in the history but exists in the current file, then it is presumed that the resource is new. If the resource was in the previous history but is different in the current commit then it is presumed that those changes need to be applied. I'm sure this works in simple use-cases. But at any moderate scale this could break down very quickly.
jacurtis 3y ago
My argument is basically this. It is a misuse of git. The author wants to eliminate state, so they suggest using git because as it turns out... you do need state. But the author didn't eliminate state at all, they just eliminated a seperate statefile. If that's what bothered them, then sure, the problem is solved. But now you are using version control as state and those don't always line up. If you only work linearly then it is perfect. But what happens when you need to start tracking a pre-existing resource? You have to import it into git history somehow. Back to your point, you really should not be modifying git history at all. There are tools to do so, but anytime you modify git history it is technically a mis-use of git. There are side-effects of doing so, especially when working on a large distributed team. Basically the author trades one minor problem for many major problems.

by tremon 3y ago

fishnchips 3y ago

Not sure if it’s misuse. Versioning your state is not a terrible idea in general. Using Git for that is kinda natural for reasonably sized states. I personally would store the state in a different repo than the infra definitions, though.

tremon 3y ago
How would you handle branching and merging of state files? Is there a valid reason to have different state files on different branches (our use case: different branches represent different DTAP phases, and hence deploy to different environments)? What would merge conflict resolution look like for these state files? What would happen if I deploy an environment from one branch that had previously been deployed from a different branch? If the answer is "you should never branch and merge state files", then the corollary is "you should never put state files in source control".
jacurtis 3y ago
This isn't versioning your state though. This is using git commit history to essentially determine state at "apply-time". Versioning your state is a good idea and is already easy to do. For example if you use the S3 backend you can enable object versioning on your bucket and then everytime the state is updated it is versioning and you can rollback state that way. It is a good practice. Similarly, you could use the local state backend, which just keeps state in a json file in your project and then you could commit that json file to version control. Now git is versioning your state. But the author is not suggesting this, they are suggesting to not have a state file at all. You look at previous commits to determine state. So for example if you create a new resource for an ec2 instance in your code and run terraform apply, how does Terraform know it is new and needs to create this ec2 instance as opposed to updating an existing one? Normally it would look at the statefile and if it isn't in there, then tf knows to create it, if it is already in state then it knows to update it and reconcile the metadata in state to the declarative state in the code. But without a statefile this can't be done. So the author looks in the git history at the previous commit. If you can find the resource block in the commit history you know what it was last set to. If it doesn't exist in the history but exists in the current file, then it is presumed that the resource is new. If the resource was in the previous history but is different in the current commit then it is presumed that those changes need to be applied. I'm sure this works in simple use-cases. But at any moderate scale this could break down very quickly.

jacurtis 3y ago

My argument is basically this. It is a misuse of git. The author wants to eliminate state, so they suggest using git because as it turns out... you do need state. But the author didn't eliminate state at all, they just eliminated a seperate statefile. If that's what bothered them, then sure, the problem is solved. But now you are using version control as state and those don't always line up. If you only work linearly then it is perfect. But what happens when you need to start tracking a pre-existing resource? You have to import it into git history somehow. Back to your point, you really should not be modifying git history at all. There are tools to do so, but anytime you modify git history it is technically a mis-use of git. There are side-effects of doing so, especially when working on a large distributed team. Basically the author trades one minor problem for many major problems.