3 ms·

Is self-service Terraform as you describe actually successful? I use Terraform frequently and I have zero confidence that this process works except for the simp

by dolni 4y ago

Is self-service Terraform as you describe actually successful? I use Terraform frequently and I have zero confidence that this process works except for the simplest changes. For one, building out anything of complexity in Terraform often involves some trial and error to make sure all the resources are wired together correctly. For two, there are some Terraform changes that can incur downtime if you just let Terraform destroy and create resources. The solution I employ for this is what I not-so-affectionately call "Terraform surgery". It usually entails standing up the new resource(s), cutting over to them, and then destroying the old ones. Sometimes I will manually remove some stuff from the state file and delete it once Terraform has performed the cutover. That stuff is possible to do in CI, but it would be inefficient and painful. To me, the biggest benefit of Terraform is managing changes to cloud resources. You know what resources are added when, why, and by whom. You have a mechanism for promoting changes between environments confidently. And if your infra ever blew up, you would be able to put stuff back.

cybrexalpha 4y ago
I've seen it work best with an embedded plan. So engineers open a PR with terraform changes, the CI runs `terraform plan`, and comments on the PR with that diff. So you can see if your terraform is valid, if you're going to be destroying things you shouldn't, etc. Larger, more complex testing is generally done by locally invoking your terraform on a testing environment. How this is setup kind of depends, but I've seen this work well for testing out complex changes before going to the PR & Plan stage.
acdha 4y ago
> For one, building out anything of complexity in Terraform often involves some trial and error to make sure all the resources are wired together correctly. This seems like misattribution: you do that everywhere when you’re doing something new - the problem is the underlying gap between the understanding of the problem which you started with and what the underlying services actually require. You can avoid most of that by using modules to reduce the amount of new structure you need to develop and static linters (tflint, tfsec, etc.) to flag things in advance. > For two, there are some Terraform changes that can incur downtime if you just let Terraform destroy and create resources. This should never be necessary due to Terraform - that’s one of the major advantages over older tools like AWS CloudFormation. If you’re seeing this as a problem where it’s not imposed by the platform (e.g. many AWS resources won’t allow you to change the name after creation) you can use lifecycle rules to avoid forcing re-creation where that makes sense or to create the replacement first & switch over before deleting the old resource. If you relocated the definitions in your code, you can tell Terraform how to migrate old state non-destructively.

by dolni 4y ago

cybrexalpha 4y ago

I've seen it work best with an embedded plan. So engineers open a PR with terraform changes, the CI runs `terraform plan`, and comments on the PR with that diff. So you can see if your terraform is valid, if you're going to be destroying things you shouldn't, etc. Larger, more complex testing is generally done by locally invoking your terraform on a testing environment. How this is setup kind of depends, but I've seen this work well for testing out complex changes before going to the PR & Plan stage.

acdha 4y ago

> For one, building out anything of complexity in Terraform often involves some trial and error to make sure all the resources are wired together correctly. This seems like misattribution: you do that everywhere when you’re doing something new - the problem is the underlying gap between the understanding of the problem which you started with and what the underlying services actually require. You can avoid most of that by using modules to reduce the amount of new structure you need to develop and static linters (tflint, tfsec, etc.) to flag things in advance. > For two, there are some Terraform changes that can incur downtime if you just let Terraform destroy and create resources. This should never be necessary due to Terraform - that’s one of the major advantages over older tools like AWS CloudFormation. If you’re seeing this as a problem where it’s not imposed by the platform (e.g. many AWS resources won’t allow you to change the name after creation) you can use lifecycle rules to avoid forcing re-creation where that makes sense or to create the replacement first & switch over before deleting the old resource. If you relocated the definitions in your code, you can tell Terraform how to migrate old state non-destructively.