3 ms·

The challenge isn't just defining multiple DNS records—it’s doing so dynamically based on an unknown number of instances at plan time. Terraform struggles with

The challenge isn't just defining multiple DNS records—it’s doing so dynamically based on an unknown number of instances at plan time. Terraform struggles with truly dynamic resource creation because it relies on a static graph. You can use count or for_each, but those require knowing the instances in advance within the Terraform configuration. If your instances are created dynamically outside Terraform (e.g., via auto-scaling groups), you hit limitations. You can work around this by using external data sources or separate workflows (e.g., running Terraform after instances are created), but that just proves the point: Terraform isn’t fully "code" in the sense of having true loops and dynamic logic like a real programming language. If you think this is trivially easy, show me how you'd do it without resorting to hacks like running terraform apply twice.

taberiand 2y ago
You can't do it if you have the instances created with auto-scaling groups of course. But nobody would think you could, that's runtime not infra. With Terraform you are supposed to have multiple coupled state files, with a tree structure of references, so that eg the state file containing the DNS can reference the previously applied state file that created the instances You are supposed to run terraform apply in a sequence that respects the dependency graph. Terragrunt makes this trivial.
- kacesensitive 2y ago
  At that point, you’re conceding the exact limitation I was pointing out. Terraform can't handle truly dynamic infrastructure changes within a single plan because its execution model is declarative, not imperative. Saying "nobody would think you could" just acknowledges that Terraform lacks the flexibility of real code—because if it were actual code, you'd be able to handle this inline rather than orchestrating multiple runs with external tools. Yes, you can manage this with separate state files and a structured apply sequence (e.g., using Terragrunt), but that’s just adding more scaffolding to work around Terraform’s inability to express dynamic logic. That’s infrastructure orchestration, not infrastructure as code. The fact that we have to rely on external tools or multi-step workflows to accomplish something that would be trivial in a real programming language just reinforces the point: Terraform isn’t really "code" in the traditional sense.
  - taberiand 2y ago
    The point is terraform can do it, and it does it well. Just because you don't want to use Terraform properly doesn't mean it's bad at what it does. Using "a real programming language" to do the infra still has to solve the same issues faced by terraform. Using a programming language to define infra doesn't solve the auto scaling DNS issue, for example, you'll be using lambdas to create the dns either way. It also doesn't inherently solve coordination of the resource deployment, you still need to organise your code into modules and ensure the order of execution. If you think Terraform is the problem here you're blaming the tool for a failure of process and understanding
    - kacesensitive 2y ago
      The core issue isn't whether Terraform can do it—it's how it does it. You're describing a workflow that requires multiple state files, external tools like Terragrunt, and sequential terraform apply runs to work around the fact that Terraform itself lacks imperative, runtime-driven logic. That’s not "using Terraform properly"—that’s compensating for its limitations. And sure, using a general-purpose language doesn’t magically eliminate coordination problems, but it does give you far more control. If I were using something like Pulumi or CDK, I wouldn’t need to hack around Terraform’s static graph by splitting state files and manually sequencing deployments. I could express logic directly in code—dynamically querying instance IDs, handling autoscaling changes, and updating DNS records inline, without requiring an entirely separate execution step. So no, this isn't a "failure of process and understanding"—it's just recognizing that Terraform’s declarative model is great for static infrastructure but falls short when dealing with truly dynamic scenarios. If you think its workflow is fine, that’s cool—but don’t pretend it doesn’t have real limitations just because you've built processes to work around them.
      - tekla 2y ago
        Yes, if you use your tools poorly, it will turn out badly. I run infra that has dynamic scaling provisioned via Terraform. Guess what, we auto-scale from hundreds to thousands of boxes a day dynamically managed through TF and we have had no issues, since we bothered to figure out how to do it properly.

taberiand 2y ago

You can't do it if you have the instances created with auto-scaling groups of course. But nobody would think you could, that's runtime not infra. With Terraform you are supposed to have multiple coupled state files, with a tree structure of references, so that eg the state file containing the DNS can reference the previously applied state file that created the instances You are supposed to run terraform apply in a sequence that respects the dependency graph. Terragrunt makes this trivial.

kacesensitive 2y ago
At that point, you’re conceding the exact limitation I was pointing out. Terraform can't handle truly dynamic infrastructure changes within a single plan because its execution model is declarative, not imperative. Saying "nobody would think you could" just acknowledges that Terraform lacks the flexibility of real code—because if it were actual code, you'd be able to handle this inline rather than orchestrating multiple runs with external tools. Yes, you can manage this with separate state files and a structured apply sequence (e.g., using Terragrunt), but that’s just adding more scaffolding to work around Terraform’s inability to express dynamic logic. That’s infrastructure orchestration, not infrastructure as code. The fact that we have to rely on external tools or multi-step workflows to accomplish something that would be trivial in a real programming language just reinforces the point: Terraform isn’t really "code" in the traditional sense.
- taberiand 2y ago
  The point is terraform can do it, and it does it well. Just because you don't want to use Terraform properly doesn't mean it's bad at what it does. Using "a real programming language" to do the infra still has to solve the same issues faced by terraform. Using a programming language to define infra doesn't solve the auto scaling DNS issue, for example, you'll be using lambdas to create the dns either way. It also doesn't inherently solve coordination of the resource deployment, you still need to organise your code into modules and ensure the order of execution. If you think Terraform is the problem here you're blaming the tool for a failure of process and understanding
  - kacesensitive 2y ago
    The core issue isn't whether Terraform can do it—it's how it does it. You're describing a workflow that requires multiple state files, external tools like Terragrunt, and sequential terraform apply runs to work around the fact that Terraform itself lacks imperative, runtime-driven logic. That’s not "using Terraform properly"—that’s compensating for its limitations. And sure, using a general-purpose language doesn’t magically eliminate coordination problems, but it does give you far more control. If I were using something like Pulumi or CDK, I wouldn’t need to hack around Terraform’s static graph by splitting state files and manually sequencing deployments. I could express logic directly in code—dynamically querying instance IDs, handling autoscaling changes, and updating DNS records inline, without requiring an entirely separate execution step. So no, this isn't a "failure of process and understanding"—it's just recognizing that Terraform’s declarative model is great for static infrastructure but falls short when dealing with truly dynamic scenarios. If you think its workflow is fine, that’s cool—but don’t pretend it doesn’t have real limitations just because you've built processes to work around them.
    - tekla 2y ago
      Yes, if you use your tools poorly, it will turn out badly. I run infra that has dynamic scaling provisioned via Terraform. Guess what, we auto-scale from hundreds to thousands of boxes a day dynamically managed through TF and we have had no issues, since we bothered to figure out how to do it properly.