3 ms·

The most confusing part of terraform for me is that terraform's view of the infrastructure is a singleton config file that is often stored in that very infrastr

The most confusing part of terraform for me is that terraform's view of the infrastructure is a singleton config file that is often stored in that very infrastructure. And then you have to share that somehow with your team and be very careful that no one gets it out of sync. Why don't cloud providers have a nice way for tools like TF to query the current state of the infra? Maybe they do and I'm doing IaC wrong?

mooreds 9mo ago
> The most confusing part of terraform for me is that terraform's view of the infrastructure is a singleton config file that is often stored in that very infrastructure. These folks also have an article about that: https://newsletter.masterpoint.io/p/how-to-bootstrap-your-state-backend-for-your-next-terraform-or-opentofu-project https://newsletter.masterpoint.io/p/how-to-bootstrap-your-st...
- bigstrat2003 9mo ago
  That article is way overkill. One should just manually create the backend storage (S3 bucket or whatever you use). No reason to faff about with the steps in the article.
  - catlifeonmars 9mo ago
    This is excellent advice. When you have a hammer… as the expression goes. It’s crazy how many times that even knowing this, I have to catch myself and step back. IaC is a contextually different way of thinking and it’s easy to get lost.
  - GowGuy47 9mo ago
    The reason to not create the bucket are because you want to ensure that you don’t have any click ops resources that you can’t track. If you manually create anything, that means it’s not in code and therefore the rest of the team doesn’t know where it lives, who created it, or when.
colechristensen 9mo ago
There are three things: * Your terraform code * The state terraform holds which is what it thinks your infrastructure state is * The actual state of your infrastructure >Why don't cloud providers have a nice way for tools like TF to query the current state of the infra? What a terraform provider is is code that queries the targeted resources through whatever APIs they provide. I guess you could argue these APIs could be better, faster, or more tuned towards infrastructure management... but gathering state from whatever resources it manages is one of the core things terraform does. I'm not sure what you're asking for.
- fragmede 9mo ago
  for the plan file to be updated to the state of the world in a non-conusing way so that apply does the right thing without a chance it's gonna blow things up.
  - colechristensen 9mo ago
    This is really up to the writer of the provider (very often the service itself) to have the provider code correctly model how the service works. It very often doesn't and allows you to plan error-free what will fail during apply. It's not an API issue but a terraform provider issue having missing or incomplete code (i.e. https://github.com/hashicorp/terraform-provider-aws https://github.com/hashicorp/terraform-provider-aws )
- akersten 9mo ago
  I want to get rid of this: > * The state terraform holds which is what it thinks your infrastructure state is Why does Terraform need that. Why can't it just call `iac.amazonaws.com/query` (or other magical endpoint) and then diff the terraform code against the actual infrastructure? I am willing to understand if the answer is "well 8 different teams work on AWS so we can't get them all to agree on how to dump their infra as JSON," but this feels like a huge (and obvious) developer experience improvement that could be made.
  - colechristensen 9mo ago
cobolexpert 9mo ago
At $WORK we have a Git repo set up by the devops team, where we can manage our junk by creating Terraform resources in our main AWS account. The state however is always stored in a _separate AWS account_ that only the devops team can manage. I find this to be a reasonable way of working with TF. I agree that it is confusing though, because one is using $PROVIDER to both create things and manage those things at the same time, but conceptually from TF’s perspective they are very different things.
don-code 9mo ago
> Why don't cloud providers have a nice way for tools like TF to query the current state of the infra? Maybe they do and I'm doing IaC wrong? This is technically how Ansible works. Here's an extensive list of modules that deploy resources in various public clouds: https://docs.ansible.com/projects/ansible/2.9/modules/list_of_cloud_modules.html https://docs.ansible.com/projects/ansible/2.9/modules/list_o... That said, it looks like Ansible has deprecated those modules, and that seems fair - I haven't actually heard of anyone deploying infrastructure in a public cloud with Ansible in years. It found its niche is image generation and systems management. Almost all modern tools like Terraform, Pulumi, and even CloudFormation (albeit under the hood) keep a state file.
- knowhy 9mo ago
  I think there are active maintained modules https://docs.ansible.com/projects/ansible/latest/collections/amazon/aws/index.html#plugins-in-amazon-aws https://docs.ansible.com/projects/ansible/latest/collections... At work we use Ansible to setup Route53 records for infrastructure hosted elsewhere. Not sure if that counts as infrastructure.
cyberax 9mo ago
> Why don't cloud providers have a nice way for tools like TF to query the current state of the infra? They do! In fact, this is my greatest pet peeve with TF, it adds state when it's not needed. I was doing infra-as-code without TF with AWS long time ago. It went like this: env_tag = "${project_name}-${env_name}" aws_instances = conn.describe_instances(filter_by_tag={"env_tag": env_tag}) if len(aws_instances) != 1: conn.launch_aws_instances(tags={"env_tag": env_tag}) AWS has tag-on-create now, making this sort of code reliable. Before that, you could do the same with instance idempotency tokens. GCP also has tags.
raffraffraff 9mo ago
There is the code, the recorded state of the infra when you applied the code and the actual state at some point in the future (which may have drifted) . You store the code in git, the recorded state (which contains unique IDs, ARNs etc) in a bucket and you read the "actual state" next time you run a plan, and you detect drift. These days people store the state in terraform cloud or spaceliftor env0 or whatever. Doesn't have to be the same infra you deployed. If you were a lunatic you could not use a state backend and just let it create state files in the terraform code directory, check the file into git with all those secrets and unique ids etc.
deleted 9mo ago
[deleted]
pjjpo 9mo ago
One big reason I tend to build on GCP instead of AWS is it's much easier to use with Terraform. GCP's APIs are generally defined as a semantic unit while AWS has ad-hoc resources that get strung together by the console or CLIs, not the APIs. An example is a k8s cluster in AWS takes a dozen resources while in GCP it's just one. While there are then third party (I think) Terraform modules to try to abstract the AWS world into an easier to use interface, they can't really solve the problem that in the end Terraform manages resources and orchestrating changes including deletion across a dozen of resources is much harder than a single one. GCP is huge so I wouldn't be surprised if there are also problematic units there with less good definition. But I would still argue that there are cloud providers that provide a reasonable view into their infra fo IAC.

mooreds 9mo ago

> The most confusing part of terraform for me is that terraform's view of the infrastructure is a singleton config file that is often stored in that very infrastructure. These folks also have an article about that: https://newsletter.masterpoint.io/p/how-to-bootstrap-your-state-backend-for-your-next-terraform-or-opentofu-project https://newsletter.masterpoint.io/p/how-to-bootstrap-your-st...

bigstrat2003 9mo ago
That article is way overkill. One should just manually create the backend storage (S3 bucket or whatever you use). No reason to faff about with the steps in the article.
- catlifeonmars 9mo ago
  This is excellent advice. When you have a hammer… as the expression goes. It’s crazy how many times that even knowing this, I have to catch myself and step back. IaC is a contextually different way of thinking and it’s easy to get lost.
- GowGuy47 9mo ago
  The reason to not create the bucket are because you want to ensure that you don’t have any click ops resources that you can’t track. If you manually create anything, that means it’s not in code and therefore the rest of the team doesn’t know where it lives, who created it, or when.

colechristensen 9mo ago

There are three things: * Your terraform code * The state terraform holds which is what it thinks your infrastructure state is * The actual state of your infrastructure >Why don't cloud providers have a nice way for tools like TF to query the current state of the infra? What a terraform provider is is code that queries the targeted resources through whatever APIs they provide. I guess you could argue these APIs could be better, faster, or more tuned towards infrastructure management... but gathering state from whatever resources it manages is one of the core things terraform does. I'm not sure what you're asking for.

fragmede 9mo ago
for the plan file to be updated to the state of the world in a non-conusing way so that apply does the right thing without a chance it's gonna blow things up.
- colechristensen 9mo ago
  This is really up to the writer of the provider (very often the service itself) to have the provider code correctly model how the service works. It very often doesn't and allows you to plan error-free what will fail during apply. It's not an API issue but a terraform provider issue having missing or incomplete code (i.e. https://github.com/hashicorp/terraform-provider-aws https://github.com/hashicorp/terraform-provider-aws )
akersten 9mo ago
I want to get rid of this: > * The state terraform holds which is what it thinks your infrastructure state is Why does Terraform need that. Why can't it just call `iac.amazonaws.com/query` (or other magical endpoint) and then diff the terraform code against the actual infrastructure? I am willing to understand if the answer is "well 8 different teams work on AWS so we can't get them all to agree on how to dump their infra as JSON," but this feels like a huge (and obvious) developer experience improvement that could be made.
- colechristensen 9mo ago

cobolexpert 9mo ago

At $WORK we have a Git repo set up by the devops team, where we can manage our junk by creating Terraform resources in our main AWS account. The state however is always stored in a _separate AWS account_ that only the devops team can manage. I find this to be a reasonable way of working with TF. I agree that it is confusing though, because one is using $PROVIDER to both create things and manage those things at the same time, but conceptually from TF’s perspective they are very different things.

don-code 9mo ago

> Why don't cloud providers have a nice way for tools like TF to query the current state of the infra? Maybe they do and I'm doing IaC wrong? This is technically how Ansible works. Here's an extensive list of modules that deploy resources in various public clouds: https://docs.ansible.com/projects/ansible/2.9/modules/list_of_cloud_modules.html https://docs.ansible.com/projects/ansible/2.9/modules/list_o... That said, it looks like Ansible has deprecated those modules, and that seems fair - I haven't actually heard of anyone deploying infrastructure in a public cloud with Ansible in years. It found its niche is image generation and systems management. Almost all modern tools like Terraform, Pulumi, and even CloudFormation (albeit under the hood) keep a state file.

knowhy 9mo ago
I think there are active maintained modules https://docs.ansible.com/projects/ansible/latest/collections/amazon/aws/index.html#plugins-in-amazon-aws https://docs.ansible.com/projects/ansible/latest/collections... At work we use Ansible to setup Route53 records for infrastructure hosted elsewhere. Not sure if that counts as infrastructure.

cyberax 9mo ago

> Why don't cloud providers have a nice way for tools like TF to query the current state of the infra? They do! In fact, this is my greatest pet peeve with TF, it adds state when it's not needed. I was doing infra-as-code without TF with AWS long time ago. It went like this: env_tag = "${project_name}-${env_name}" aws_instances = conn.describe_instances(filter_by_tag={"env_tag": env_tag}) if len(aws_instances) != 1: conn.launch_aws_instances(tags={"env_tag": env_tag}) AWS has tag-on-create now, making this sort of code reliable. Before that, you could do the same with instance idempotency tokens. GCP also has tags.

raffraffraff 9mo ago

There is the code, the recorded state of the infra when you applied the code and the actual state at some point in the future (which may have drifted) . You store the code in git, the recorded state (which contains unique IDs, ARNs etc) in a bucket and you read the "actual state" next time you run a plan, and you detect drift. These days people store the state in terraform cloud or spaceliftor env0 or whatever. Doesn't have to be the same infra you deployed. If you were a lunatic you could not use a state backend and just let it create state files in the terraform code directory, check the file into git with all those secrets and unique ids etc.

deleted 9mo ago

[deleted]

pjjpo 9mo ago

One big reason I tend to build on GCP instead of AWS is it's much easier to use with Terraform. GCP's APIs are generally defined as a semantic unit while AWS has ad-hoc resources that get strung together by the console or CLIs, not the APIs. An example is a k8s cluster in AWS takes a dozen resources while in GCP it's just one. While there are then third party (I think) Terraform modules to try to abstract the AWS world into an easier to use interface, they can't really solve the problem that in the end Terraform manages resources and orchestrating changes including deletion across a dozen of resources is much harder than a single one. GCP is huge so I wouldn't be surprised if there are also problematic units there with less good definition. But I would still argue that there are cloud providers that provide a reasonable view into their infra fo IAC.