3 ms·

My current employer (BigCo) has Terraform managing both infra + deployments in Terraform, at (ludicrous) scale. It's a nightmare. The problem with Terraform is

My current employer (BigCo) has Terraform managing both infra + deployments in Terraform, at (ludicrous) scale. It's a nightmare. The problem with Terraform is that you must plan your workspaces such that you will not exceed the best-practice amount of resources per workspace (~100-200) or else plans will drastically slow down your time-to-deploy, checking stuff like databases and networking that you haven't touched and have no desire to touch. In practice this means creating a latticework of Terraform workspaces that trigger each other, and there are currently no good open-source tools that support it. Best practice as I can currently see it is to have Terraform set up what you need for continuous delivery (e.g. ArgoCD) as part of the infrastructure, then use the CD tool to handle day-to-day deployments. Most CD tooling then asks you to package your deployment in something like Helm.

chucky_z 2y ago
You can setup dependent stacks in CDKTF. It’s far from as clean as a standard TF DAG plan/apply but I’m having a lot of success with it right now. If I were actively using k8s at the moment I would probably setup dependent cluster resources using this method, e.g: ensure a clean, finished CSI daemon deployment before deploying a deployment using that CSI provider :)
- solatic 2y ago
  You're right that CDKTF with dependent stacks is probably better than nothing, but (a) CDKTF's compatibility with OpenTofu depends on a lack of breaking changes in CDKTF, since the OpenTofu team didn't fork CDKTF, so this is a little hairy for building production infrastructure; (b) CDKTF stacks, even when they can run in parallel, still run on the same machine that invoked CDKTF. When you have (ludicrous scale) X number of "stacks", this isn't a good fit. It's something that should be doable in one of the managed Terraform services, but the pricing if you try to do (ludicrous scale) parallelism gets to be insane.

chucky_z 2y ago

You can setup dependent stacks in CDKTF. It’s far from as clean as a standard TF DAG plan/apply but I’m having a lot of success with it right now. If I were actively using k8s at the moment I would probably setup dependent cluster resources using this method, e.g: ensure a clean, finished CSI daemon deployment before deploying a deployment using that CSI provider :)

solatic 2y ago
You're right that CDKTF with dependent stacks is probably better than nothing, but (a) CDKTF's compatibility with OpenTofu depends on a lack of breaking changes in CDKTF, since the OpenTofu team didn't fork CDKTF, so this is a little hairy for building production infrastructure; (b) CDKTF stacks, even when they can run in parallel, still run on the same machine that invoked CDKTF. When you have (ludicrous scale) X number of "stacks", this isn't a good fit. It's something that should be doable in one of the managed Terraform services, but the pricing if you try to do (ludicrous scale) parallelism gets to be insane.