7 ms·

After some work with kubernetes, i must really say, helm is a complexity hell. I'm sure it has much features but many aren't needed but increase the complexity

by buster 11mo ago

After some work with kubernetes, i must really say, helm is a complexity hell. I'm sure it has much features but many aren't needed but increase the complexity nonetheless. Also, please fix the "default" helm chart template, it's a nightmare of options and values no beginner understands. Make it basic and simple. Nowadays i would very much prefer to just use terraform for kubernetes deployments, especially if you use terraform anyway!

nullwarp 11mo ago
I don't think I've ever seen a Helm template that didn't invoke nightmares. Probably the biggest reason I moved away from Kubernetes in the first place.
- bigstrat2003 11mo ago
  We have several Helm charts we've written at my job and they are very pleasant to use. They are just normal k8s templates with a couple of values parameterized, and they work great. The ones people put out for public consumption are very complex, but it isn't like Helm charts have to be that complex.
  - phyrog 11mo ago
    In my book the main problem with Helm charts is that every customization option needs to be implemented by the chart that way. There is no way for chart consumer to change anything the chart author did not allow to be changed. That leads to these overly complex and config heavy charts people publish - just to make sure everything is customizable for consumers. I'd love something that works more like Kustomize but with other benefits of Helm charts (packaging, distribution via OCI, more straight forward value interpolation than overlays and patches, ...). So far none have ticked all my boxes.
    - glotzerhotze 11mo ago
      fluxCD brings a really nice helm-controller that will allow to change manifests via a postRenderers stub while still allowing to use regular helm tooling against the cluster. https://fluxcd.io/flux/components/helm/helmreleases/#post-renderers https://fluxcd.io/flux/components/helm/helmreleases/#post-re...
      - phyrog 11mo ago
        Yeah, but then it is yet another layer of configuration slapped on top of the previous layer of configuration. That can't be the best solution, can it? Same thing for piping helm template through Kustomize.
        maherbeg 11mo ago
        Yeah, this setup is both nice and insane. If you don't need much extra customization it's great. But I have a setup where I needed both postBuild and postRenderer's + actual kustomization layering and it was awful trying to figure out the order of execution to get the right final output. In hindsight it would have been much faster to write the resources myself.
        nwmcsween 11mo ago
        Use helm to generate the manifests with a Makefile, use Kustomize to change said manifests for prod, staging, etc.
    - lillecarl 11mo ago
      Kustomize can render Helm charts. It's "very basic" as in Kustomize will call the Helm binary to render the template, ingest it and apply patches. I wrote a tool called "easykubenix" that works in a similar way, render the chart in a derivation, convert the YAML to JSON, import JSON into the Nix module structure and now you're free to override, remove or add anything you want :) It's still very CLI deploy centric using kluctl as the deployment engine, but there's nothing preventing dumping the generated JSON (or YAML) manifests into a GitOps loop. It doesn't make the public charts you consume any less horrible, but you don't have to care as much about them at least
    - ranger207 11mo ago
      Yeah too many times the Helm chart is barely less complex than writing all the manifests yourself because all the manifest options are still in the chart
  - honkycat 11mo ago
    Yes, this is the key. Helm charts should basically be manifests with some light customization. Helm is not good enough to develop abstractions with. So go the opposite way: keep it stupid simple. Pairing helm with Kustomize can help a lot as well. You do most of the templating in the helm chart but you have an escape hatch if you need more patches.
  - cogman10 11mo ago
    That's generally what I try to push for in my company. A single purpose chart for your project is generally a lot easier to grok and consume vs what can be done. I think the likes of "kustomize" is probably a more sane route to go down. But our entire infrastructure is already helm so hard to switch that all out.
    - bigstrat2003 11mo ago
      I'm ashamed to say it but I cannot for the life of me understand how kustomize works. I could not ever figure out how to do things outside the "hello world" tutorials they walk you through. I'm not a stupid person (citation needed lol), but trying to understand the kustomize docs made me feel incredibly stupid. That's why we didn't go with that instead of Helm.
      - globular-toast 11mo ago
        Helm requires you to write a template and you need to know (or guess) up front which values you want to be configurable. Then you set sane defaults for those values. If you find a user needs to change something else you have to edit the chart to add it. With Kustomize, on the other hand, you just write the default as perfectly normal K8s manifests in YAML. You don't have to know or care what your users are going to do with it. Then you write a `kustomizatiom.yaml` that references those manifests somehow (could be in the same folder or you can use a URL). Kustomize simply concatenates everything together as its default behaviour. Run `kubectl kustomize` in the directory with `kustomization.yaml` to see the output. You can run `kubectl apply -k` to apply to your cluster (and `kubectl delete -k` to delete it all). From there you just add what you need to `kustomization.yaml`. You can do a few basics easily like setting the namespace for it all, adding labels to everything and changing the image ref. Keep running `kubectl kustomize` to see how it's changing things. You can use configmap and secret generators to easily generate these with hashed names and it will make sure all references match the generated name. Then you have the all powerful YAML or JSON editing commands which allow you to selectively edit the manifests if you need to. Start small and add things when you need them. Keep running `kubectl kustomize` at every step until you get it.
    - 11mo ago
  - brainzap 11mo ago
    this, our helm charts are flat and for year only passed in the image as variable
- dev_l1x_be 11mo ago
  What did you move to?
verdverm 11mo ago
Helm is my example of where DevOps lost it's way. The insanity of multiple tiers on templating an invisible char scoped language... it blows my mind that so many of us just deal with it Nowadays I'm using CUE in front of TF & k8s, in part because I have workloads that need a bit of both and share config. I emit tf.json and Yaml as needed from a single source of truth
- pjmlp 11mo ago
  The problem with Kubernetes, Docker and anything CNCF related is what happens when everyone and their dog tries to make a business out of an OS capability with venture capital.
- mkroman 11mo ago
  shudders.. `| nindent 12`.. I've been trying to apply CUE to my work, but the tooling just isn't there for much of what I need yet. It also seems really short-sighted that it is implemented in Go which is notoriously bad for embedding.
  - hvenev 11mo ago
    Back when my job involved using Kubernetes and Helm, the solution I found was to use `| toJson` instead: it generates one line that happens to be valid YAML as well.
  - verdverm 11mo ago
    > seems really short-sighted that it is implemented in Go CUE was a fork of the Go compiler (Marcel was on the Go team at the time and wanted to reuse much of the infra within the codebase) Also, so much of the k8s ecosystem is in Go that it was a natural choice.
    - mkroman 11mo ago
      > CUE was a fork of the Go compiler (Marcel was on the Go team at the time and wanted to reuse much of the infra within the codebase) Ah, that makes sense, I guess. I also get the feeling that the language itself is still under very active development, so until 1.0 is released I don't think it matters too much what it's implemented in. > Also, so much of the k8s ecosystem is in Go that it was a natural choice. That might turn out to be a costly decision, imho. I wanted to use CUE to manage a repository of schema definitions, and from these I wanted to generate other formats, such as JSON schemas, with constraints hopefully taken from the high-level CUE. I figured I'd try and hack something together, but it was a complete non-starter since I don't work within the Go ecosystem. Projects like the cue language live and breathe from an active community with related tooling, so the decision still really boggles my mind. I'll stay optimistic and hope that once it reaches 1.0, someone will write an implementation that is easily embedded for my use-cases. I won't hold my breath though, since the scope is getting quite big.
timiel 11mo ago
Do you have any resources regarding using tf to handle deployments ? I’d love to dig a bit.
- Aeolun 11mo ago
  The kubernetes provider mostly just works exactly as you expect
- buster 11mo ago
  Just use https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/guides/getting-started.html https://registry.terraform.io/providers/hashicorp/kubernetes... instead of helm...
  - Traubenfuchs 11mo ago
    …but how do you install helm charts via terraform? Is there a helm provider? If not, what would be the right way to install messy stuff like nginx ingress, cert-manager, etc.?
    - buster 11mo ago
      There is a helm provider. Why would you need it? Can't you just use the kubernetes provider? People probably don't realize, that helm mostly is templating for the YAMLs kubernetes wants (plus a lot of other stuff that increases complexity).
      - vbezhenar 11mo ago
        There are many applications which are distributed as helm charts. Those charts install multiple deployments, service accounts and whatnot. They barely document all these things. So if you want to avoid helm, you gotta do a whole lot of reverse-engineering. You gotta render a chart, explore all the manifests, explore all the configuration options, find out if they're needed or not. An alternative is to just use helm, invoking it and forgetting about it. You can't blame people for going the easy way, I guess...
jadbox 11mo ago
I don't think I want to use kubernetes (or anything that uses it) again. Nightmare of broken glass. Back in the day Docker Compose gave me 95% of what I wanted and the complexity was basically one file with few surprises.
- lxe 11mo ago
  Docker Compose still takes you 95% of what you need. I wish Docker Swarm survived.
  - Alir3z4 11mo ago
    What happened to it? I'm still using it with not a single issue (except when is messes up the iptables rules) I still confidently, upgrade the docker across all the nodes, workers and managers and it just works. Not a single time that it caused an issue.
    - lxe 11mo ago
      For some reason I assumed it was unsupported. That doesn't seem to be the case.
      - Cyphus 11mo ago
        The original iteration of Docker Swarm, now known as Classic, is deprecated. Maybe you were thinking of that?
        lxe 11mo ago
        As I read more about it, yes, that is indeed the case.
    - Cyphus 11mo ago
      Docker the company bet big on Swarm being the de facto container orchestration platform for businesses. It just got completely overshadowed by k8s. Swarm continues to exist and be actively developed, but it’s doomed to fade into obscurity.
lxe 11mo ago
Infrastructure as code should from the beginning have been through a strict typed language with solid dependency and packaging contract. I know that there are solutions like CDK and SST that attempt this, but because the underlying mechanisms are not native to those solutions, it's simply not enough, and the resulting interfaces are still way too brittle and complex.
- JohnMakin 11mo ago
  I mean terraform provides this but using it doesn't give a whole lot of value, at least IME. I enforce types but often an upstream provider implementation will break that convention. It's rarely the fault of the IAC itself and usually the fault of the upstream service when things get annoying.
- contrahax 11mo ago
  Try pulumi!
  - lxe 11mo ago
    Yup that's what SST wraps (or at least it did when I was fiddling with it). And even Pulumi still is at the behest of the cloud providers... it still has to mirror complexity of the providers to a considerable degree. Devexp is leaps and bounds better with Pulumi than CDK though.
dev_l1x_be 11mo ago
Could you explain this a bit? Is helm optional part of the k8s stack?
- pests 11mo ago
  Helm is not official or blessed or anything, just another third party tool people install after install k8s.
- JamesSwift 11mo ago
  Helm is sort of like a docker (or maybe docker compose) for k8s, in terms of a helm chart is a prepackaged k8s "application" that you can ship to your cluster. It got very popular very quickly because of the ease of use, and I think that was premature which affects its day-to-day usability.
  - deleted 11mo ago
    [deleted]
- buster 11mo ago
  Yes, you really don't need to use helm if you have terraform. Just use https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/guides/getting-started.html https://registry.terraform.io/providers/hashicorp/kubernetes... . If you used helm + terraform before, you'll have no problem understanding the terraform kubernetes provider (as opposed to the helm provider).
  - e12e 11mo ago
    It does make it challenging to track operators as upstream usually only provide/document helm installation. If you write your own tf definition of operator x v1, it can be tricky to upgrade to v2 - as you need to figure out what changes are needed in your tf config to go from v1 to v2.
- mx_03 11mo ago
  The way I understand, helm is the npm of k8s. You can install, update, and remove an app in your k8s cluster using helm. And you release a new version of your app to a helm repository.
Hamuko 11mo ago
Incidentally, Terraform is the only way I want to use Helm at all. Although the Terraform provider for Helm is quite cumbersome to use when you need to set values.
ctm92 11mo ago
Kustomize with ArgoCD is my go to
vbezhenar 11mo ago
I've embraced kustomize and I like it. It's simple enough and powerful enough for my needs. A bit verbose to type out all the manifests, but I can live with it.
- natebc 11mo ago
  This is what I've done too. Just enough features easily available to handle everything i've ever needed in the simple deployments I use. Secrets, A/B configuration, even "dynamic reload" of a Deployment for Configmap changes. Gets the job done.
- pyrale 11mo ago
  I'm using sed on my yaml files. Currently considering kustomize instead, but I wouldn't touch Helm with a 10 foot pole.
Glamklo 11mo ago
[dead]
e12e 11mo ago
I only whish terraform was more recognized by upstream projects, like postgres, tailscale, ingress operators. A one-time adoption from kubectl yaml or helm to terraform is doable - but syncing upstream updates is a chore. If terraform (or another rich format) was popular as source of truth - then perhaps helm and kubectl yaml could be built from a terraform definition, with benefits like variable documentation, validation etc.

by buster 11mo ago

nullwarp 11mo ago

I don't think I've ever seen a Helm template that didn't invoke nightmares. Probably the biggest reason I moved away from Kubernetes in the first place.

bigstrat2003 11mo ago
We have several Helm charts we've written at my job and they are very pleasant to use. They are just normal k8s templates with a couple of values parameterized, and they work great. The ones people put out for public consumption are very complex, but it isn't like Helm charts have to be that complex.
- phyrog 11mo ago
  In my book the main problem with Helm charts is that every customization option needs to be implemented by the chart that way. There is no way for chart consumer to change anything the chart author did not allow to be changed. That leads to these overly complex and config heavy charts people publish - just to make sure everything is customizable for consumers. I'd love something that works more like Kustomize but with other benefits of Helm charts (packaging, distribution via OCI, more straight forward value interpolation than overlays and patches, ...). So far none have ticked all my boxes.
  - glotzerhotze 11mo ago
    fluxCD brings a really nice helm-controller that will allow to change manifests via a postRenderers stub while still allowing to use regular helm tooling against the cluster. https://fluxcd.io/flux/components/helm/helmreleases/#post-renderers https://fluxcd.io/flux/components/helm/helmreleases/#post-re...
    - phyrog 11mo ago
      Yeah, but then it is yet another layer of configuration slapped on top of the previous layer of configuration. That can't be the best solution, can it? Same thing for piping helm template through Kustomize.
      - maherbeg 11mo ago
        Yeah, this setup is both nice and insane. If you don't need much extra customization it's great. But I have a setup where I needed both postBuild and postRenderer's + actual kustomization layering and it was awful trying to figure out the order of execution to get the right final output. In hindsight it would have been much faster to write the resources myself.
      - nwmcsween 11mo ago
        Use helm to generate the manifests with a Makefile, use Kustomize to change said manifests for prod, staging, etc.
  - lillecarl 11mo ago
    Kustomize can render Helm charts. It's "very basic" as in Kustomize will call the Helm binary to render the template, ingest it and apply patches. I wrote a tool called "easykubenix" that works in a similar way, render the chart in a derivation, convert the YAML to JSON, import JSON into the Nix module structure and now you're free to override, remove or add anything you want :) It's still very CLI deploy centric using kluctl as the deployment engine, but there's nothing preventing dumping the generated JSON (or YAML) manifests into a GitOps loop. It doesn't make the public charts you consume any less horrible, but you don't have to care as much about them at least
  - ranger207 11mo ago
    Yeah too many times the Helm chart is barely less complex than writing all the manifests yourself because all the manifest options are still in the chart
- honkycat 11mo ago
  Yes, this is the key. Helm charts should basically be manifests with some light customization. Helm is not good enough to develop abstractions with. So go the opposite way: keep it stupid simple. Pairing helm with Kustomize can help a lot as well. You do most of the templating in the helm chart but you have an escape hatch if you need more patches.
- cogman10 11mo ago
  That's generally what I try to push for in my company. A single purpose chart for your project is generally a lot easier to grok and consume vs what can be done. I think the likes of "kustomize" is probably a more sane route to go down. But our entire infrastructure is already helm so hard to switch that all out.
  - bigstrat2003 11mo ago
    I'm ashamed to say it but I cannot for the life of me understand how kustomize works. I could not ever figure out how to do things outside the "hello world" tutorials they walk you through. I'm not a stupid person (citation needed lol), but trying to understand the kustomize docs made me feel incredibly stupid. That's why we didn't go with that instead of Helm.
    - globular-toast 11mo ago
      Helm requires you to write a template and you need to know (or guess) up front which values you want to be configurable. Then you set sane defaults for those values. If you find a user needs to change something else you have to edit the chart to add it. With Kustomize, on the other hand, you just write the default as perfectly normal K8s manifests in YAML. You don't have to know or care what your users are going to do with it. Then you write a `kustomizatiom.yaml` that references those manifests somehow (could be in the same folder or you can use a URL). Kustomize simply concatenates everything together as its default behaviour. Run `kubectl kustomize` in the directory with `kustomization.yaml` to see the output. You can run `kubectl apply -k` to apply to your cluster (and `kubectl delete -k` to delete it all). From there you just add what you need to `kustomization.yaml`. You can do a few basics easily like setting the namespace for it all, adding labels to everything and changing the image ref. Keep running `kubectl kustomize` to see how it's changing things. You can use configmap and secret generators to easily generate these with hashed names and it will make sure all references match the generated name. Then you have the all powerful YAML or JSON editing commands which allow you to selectively edit the manifests if you need to. Start small and add things when you need them. Keep running `kubectl kustomize` at every step until you get it.
  - 11mo ago
- brainzap 11mo ago
  this, our helm charts are flat and for year only passed in the image as variable
dev_l1x_be 11mo ago
What did you move to?

verdverm 11mo ago

Helm is my example of where DevOps lost it's way. The insanity of multiple tiers on templating an invisible char scoped language... it blows my mind that so many of us just deal with it Nowadays I'm using CUE in front of TF & k8s, in part because I have workloads that need a bit of both and share config. I emit tf.json and Yaml as needed from a single source of truth

pjmlp 11mo ago
The problem with Kubernetes, Docker and anything CNCF related is what happens when everyone and their dog tries to make a business out of an OS capability with venture capital.
mkroman 11mo ago
shudders.. `| nindent 12`.. I've been trying to apply CUE to my work, but the tooling just isn't there for much of what I need yet. It also seems really short-sighted that it is implemented in Go which is notoriously bad for embedding.
- hvenev 11mo ago
  Back when my job involved using Kubernetes and Helm, the solution I found was to use `| toJson` instead: it generates one line that happens to be valid YAML as well.
- verdverm 11mo ago
  > seems really short-sighted that it is implemented in Go CUE was a fork of the Go compiler (Marcel was on the Go team at the time and wanted to reuse much of the infra within the codebase) Also, so much of the k8s ecosystem is in Go that it was a natural choice.
  - mkroman 11mo ago
    > CUE was a fork of the Go compiler (Marcel was on the Go team at the time and wanted to reuse much of the infra within the codebase) Ah, that makes sense, I guess. I also get the feeling that the language itself is still under very active development, so until 1.0 is released I don't think it matters too much what it's implemented in. > Also, so much of the k8s ecosystem is in Go that it was a natural choice. That might turn out to be a costly decision, imho. I wanted to use CUE to manage a repository of schema definitions, and from these I wanted to generate other formats, such as JSON schemas, with constraints hopefully taken from the high-level CUE. I figured I'd try and hack something together, but it was a complete non-starter since I don't work within the Go ecosystem. Projects like the cue language live and breathe from an active community with related tooling, so the decision still really boggles my mind. I'll stay optimistic and hope that once it reaches 1.0, someone will write an implementation that is easily embedded for my use-cases. I won't hold my breath though, since the scope is getting quite big.

timiel 11mo ago

Do you have any resources regarding using tf to handle deployments ? I’d love to dig a bit.

Aeolun 11mo ago
The kubernetes provider mostly just works exactly as you expect
buster 11mo ago
Just use https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/guides/getting-started.html https://registry.terraform.io/providers/hashicorp/kubernetes... instead of helm...
- Traubenfuchs 11mo ago
  …but how do you install helm charts via terraform? Is there a helm provider? If not, what would be the right way to install messy stuff like nginx ingress, cert-manager, etc.?
  - buster 11mo ago
    There is a helm provider. Why would you need it? Can't you just use the kubernetes provider? People probably don't realize, that helm mostly is templating for the YAMLs kubernetes wants (plus a lot of other stuff that increases complexity).
    - vbezhenar 11mo ago
      There are many applications which are distributed as helm charts. Those charts install multiple deployments, service accounts and whatnot. They barely document all these things. So if you want to avoid helm, you gotta do a whole lot of reverse-engineering. You gotta render a chart, explore all the manifests, explore all the configuration options, find out if they're needed or not. An alternative is to just use helm, invoking it and forgetting about it. You can't blame people for going the easy way, I guess...

jadbox 11mo ago

I don't think I want to use kubernetes (or anything that uses it) again. Nightmare of broken glass. Back in the day Docker Compose gave me 95% of what I wanted and the complexity was basically one file with few surprises.

lxe 11mo ago
Docker Compose still takes you 95% of what you need. I wish Docker Swarm survived.
- Alir3z4 11mo ago
  What happened to it? I'm still using it with not a single issue (except when is messes up the iptables rules) I still confidently, upgrade the docker across all the nodes, workers and managers and it just works. Not a single time that it caused an issue.
  - lxe 11mo ago
    For some reason I assumed it was unsupported. That doesn't seem to be the case.
    - Cyphus 11mo ago
      The original iteration of Docker Swarm, now known as Classic, is deprecated. Maybe you were thinking of that?
      - lxe 11mo ago
        As I read more about it, yes, that is indeed the case.
  - Cyphus 11mo ago
    Docker the company bet big on Swarm being the de facto container orchestration platform for businesses. It just got completely overshadowed by k8s. Swarm continues to exist and be actively developed, but it’s doomed to fade into obscurity.

lxe 11mo ago

Infrastructure as code should from the beginning have been through a strict typed language with solid dependency and packaging contract. I know that there are solutions like CDK and SST that attempt this, but because the underlying mechanisms are not native to those solutions, it's simply not enough, and the resulting interfaces are still way too brittle and complex.

JohnMakin 11mo ago
I mean terraform provides this but using it doesn't give a whole lot of value, at least IME. I enforce types but often an upstream provider implementation will break that convention. It's rarely the fault of the IAC itself and usually the fault of the upstream service when things get annoying.
contrahax 11mo ago
Try pulumi!
- lxe 11mo ago
  Yup that's what SST wraps (or at least it did when I was fiddling with it). And even Pulumi still is at the behest of the cloud providers... it still has to mirror complexity of the providers to a considerable degree. Devexp is leaps and bounds better with Pulumi than CDK though.

dev_l1x_be 11mo ago

Could you explain this a bit? Is helm optional part of the k8s stack?

pests 11mo ago
Helm is not official or blessed or anything, just another third party tool people install after install k8s.
JamesSwift 11mo ago
Helm is sort of like a docker (or maybe docker compose) for k8s, in terms of a helm chart is a prepackaged k8s "application" that you can ship to your cluster. It got very popular very quickly because of the ease of use, and I think that was premature which affects its day-to-day usability.
- deleted 11mo ago
  [deleted]
buster 11mo ago
Yes, you really don't need to use helm if you have terraform. Just use https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/guides/getting-started.html https://registry.terraform.io/providers/hashicorp/kubernetes... . If you used helm + terraform before, you'll have no problem understanding the terraform kubernetes provider (as opposed to the helm provider).
- e12e 11mo ago
  It does make it challenging to track operators as upstream usually only provide/document helm installation. If you write your own tf definition of operator x v1, it can be tricky to upgrade to v2 - as you need to figure out what changes are needed in your tf config to go from v1 to v2.
mx_03 11mo ago
The way I understand, helm is the npm of k8s. You can install, update, and remove an app in your k8s cluster using helm. And you release a new version of your app to a helm repository.

Hamuko 11mo ago

Incidentally, Terraform is the only way I want to use Helm at all. Although the Terraform provider for Helm is quite cumbersome to use when you need to set values.

ctm92 11mo ago

Kustomize with ArgoCD is my go to

vbezhenar 11mo ago

I've embraced kustomize and I like it. It's simple enough and powerful enough for my needs. A bit verbose to type out all the manifests, but I can live with it.

natebc 11mo ago
This is what I've done too. Just enough features easily available to handle everything i've ever needed in the simple deployments I use. Secrets, A/B configuration, even "dynamic reload" of a Deployment for Configmap changes. Gets the job done.
pyrale 11mo ago
I'm using sed on my yaml files. Currently considering kustomize instead, but I wouldn't touch Helm with a 10 foot pole.

Glamklo 11mo ago

[dead]

e12e 11mo ago

I only whish terraform was more recognized by upstream projects, like postgres, tailscale, ingress operators. A one-time adoption from kubectl yaml or helm to terraform is doable - but syncing upstream updates is a chore. If terraform (or another rich format) was popular as source of truth - then perhaps helm and kubectl yaml could be built from a terraform definition, with benefits like variable documentation, validation etc.