5 ms·

Doesn’t terraform show you how many resources are affected and require you to approve that when running apply?

knownjorbist 5y ago
It does, but according to the post it was run via Azure DevOps. I'm not familiar with ADO but it sounds like it might not have been as obvious as running locally. Alternatively, perhaps it would have only shown deleting the Resource Group, but internally to Azure(and unbeknownst to Terraform) this means deleting everything within the RG as well.
- rjbwork 5y ago
  We run our Terraform pipeline via Azure DevOps. We separate the plan/apply steps, stash both the binary and json version of the plan, stash the logs of the plan, and require increasing levels of approval as it moves through each environment on it's way from dev to prod. To go to prod it needs 4 people to have reviewed the plan - the person whose PR triggered the release, a director, a team lead, and another engineer. This is enforced using Azure DevOps' baked in pipelines and approval functionalities. To keep from nuking things we completely disable destructive operations from our AWS/Azure Terraform user and in fact, limit what the user can do as much as possible, only giving it the permissions needed to create what we use, and adding as necessary. For deletes we add a very targeted permission that we revoke after deployment. It can seem onerous at times, but ultimately it's a far smoother process than legacy manual or scripted deployments and is very repeatable and visible. It's also a good thing to have lots of eyes on critical infrastructure changes anyway.
  - pram 5y ago
    Why stop at 4 people though, and only director level?
    - rjbwork 5y ago
      I realize you're being sarcastic but we're contractually and lawfully obligated to meet that.
arlk 5y ago
.. unless you add the `-auto-approve` flag.
vimwizard 5y ago
What happened is OP imported an AzureRM resource group into a terraform resource, then "deleted" the resource by changing to a data block. When you delete a resource group in AzureRM it deletes every resource under it. This behavior is pretty stupid when mixed with terraform but as a rule of thumb if I'm not importing every resource under a resource group into the state then I make it a data call from the start. The missing step was to `terraform state rm <addr>` on the resource group and then change it to a data block.
- takeda 5y ago
  Sounds to me like a bug in azure provider. It's job supposed to be understanding the environment and knowing what will happen for any given action.
  - mattbnr32 5y ago
    AFAIK terraform doesn't have a way to inject user interaction on provider-level operations. Anyone know if az cli or powershell module prompts users before it deletes a resource group?
    - takeda 5y ago
      I didn't mean injecting an interaction. What it supposed to do is when was about to remove resource group, it would still see that other resources are relying to it and not to remove it.

It does, but according to the post it was run via Azure DevOps. I'm not familiar with ADO but it sounds like it might not have been as obvious as running locally. Alternatively, perhaps it would have only shown deleting the Resource Group, but internally to Azure(and unbeknownst to Terraform) this means deleting everything within the RG as well.

rjbwork 5y ago
We run our Terraform pipeline via Azure DevOps. We separate the plan/apply steps, stash both the binary and json version of the plan, stash the logs of the plan, and require increasing levels of approval as it moves through each environment on it's way from dev to prod. To go to prod it needs 4 people to have reviewed the plan - the person whose PR triggered the release, a director, a team lead, and another engineer. This is enforced using Azure DevOps' baked in pipelines and approval functionalities. To keep from nuking things we completely disable destructive operations from our AWS/Azure Terraform user and in fact, limit what the user can do as much as possible, only giving it the permissions needed to create what we use, and adding as necessary. For deletes we add a very targeted permission that we revoke after deployment. It can seem onerous at times, but ultimately it's a far smoother process than legacy manual or scripted deployments and is very repeatable and visible. It's also a good thing to have lots of eyes on critical infrastructure changes anyway.
- pram 5y ago
  Why stop at 4 people though, and only director level?
  - rjbwork 5y ago
    I realize you're being sarcastic but we're contractually and lawfully obligated to meet that.

arlk 5y ago

.. unless you add the `-auto-approve` flag.

vimwizard 5y ago

What happened is OP imported an AzureRM resource group into a terraform resource, then "deleted" the resource by changing to a data block. When you delete a resource group in AzureRM it deletes every resource under it. This behavior is pretty stupid when mixed with terraform but as a rule of thumb if I'm not importing every resource under a resource group into the state then I make it a data call from the start. The missing step was to `terraform state rm <addr>` on the resource group and then change it to a data block.

takeda 5y ago
Sounds to me like a bug in azure provider. It's job supposed to be understanding the environment and knowing what will happen for any given action.
- mattbnr32 5y ago
  AFAIK terraform doesn't have a way to inject user interaction on provider-level operations. Anyone know if az cli or powershell module prompts users before it deletes a resource group?
  - takeda 5y ago
    I didn't mean injecting an interaction. What it supposed to do is when was about to remove resource group, it would still see that other resources are relying to it and not to remove it.