5 ms·
> This means the Service must be created first, then when the corresponding NEG is created the name can be queried and added to the Terraform project. These k
by rossmohax 7y ago
> This means the Service must be created first, then when the corresponding NEG is created the name can be queried and added to the Terraform project.
These kind of reverse dependencies, where app level changes have to be reflected in infrastructure are source of endless bugs and headaches.
To do it right, you'd want to encode app and infra changes in the same ubiquitous tool, but "infrastructure as a code" tools such as terraform suck big times at app level deploys.
Pulumi takes steps in right direction, where it is actually not that painfull to manage everything, that is what I ended up using for very similar problem configuring GCPs load balancers for my k8s apps.
- cbushko 7y agoWe use terraform for our kubernetes app configuration and it works well. Being able to pass values from one resource to another is extremely powerful. For example, the database instance credentials down into cloudsql proxy. The only and most frustrating missing piece of the terraform kubernetes provider is CRDs. I hacked around it by using some yaml provider bit I would like a cleaner solution.
- rossmohax 7y agoTF kubernetes provider works for simplest cases only. How would you wait for PV to be created, when your TF resources specify PVC? That is to help users to diagnose problems with deployments.
- cppforlife 7y agosomeone asked me to wrap get-kapp.io (one of my projects) with terraform so that they can pass down infra settings to their k8s config. i ended up writing https://githubc.com/k14s/terraform-provider-k14s https://githubc.com/k14s/terraform-provider-k14s (example: https://github.com/k14s/terraform-provider-k14s/tree/master/examples/guestbook https://github.com/k14s/terraform-provider-k14s/tree/master/...). i havent been asked about case where app info is needed by terraform, but may be thats something to add...
- solatic 7y ago> To do it right, you'd want to encode app and infra changes in the same ubiquitous tool Well, that's exactly what we do with Dhall, which lets us put static types on infrastructure tooling that prints out unpredictable output but in a predictable format and transform that output into the format that other dependent tools expect. Everything is then glued together with simple shell scripts and run in a CI environment. Once you understand the pattern - use an uber-config plus typed tool outputs to generate downstream config and apply all config in an idempotent way - you wonder why you ever did things any other way.
- rossmohax 7y agoI looked at Dhall before. It is a pure language for static configs, right? It can't help me to express "deploy k8s resource, wait for annotation to appear, take annotation value and use it to deploy infra resource". Wrapping multiple tools in a bash is exact thing I wish I didn't have to do :( If Dhall enables safer bash glue, I'd be happy to read how
- hinkley 7y agoWhat about the Pulumi solution gets in your way?
- rossmohax 7y agoNo tools IaC tools I know of including Pulumi, allow to describe evolutions for infra. Simple use case - deploy canary instance, check it's health, scale down canary instance, update main pool of instances. In other words, there is no way to express sequence of changes of the same object in the single run of IaC tool. We are forced to wrap IaC tools in layers of bash to simulate it.
- hinkley 7y agoAh, good to know, thanks. What you're describing sounds a bit like migrations for databases. Do you agree?