4 ms·

Helm, in my personal opinion, is one of the biggest mistakes of the kubernetes ecosystem. Not embracing Jsonnet, Cue, Dhall, or even an in-language DSL like Pul

Helm, in my personal opinion, is one of the biggest mistakes of the kubernetes ecosystem. Not embracing Jsonnet, Cue, Dhall, or even an in-language DSL like Pulumi has set the usability of kube back quite far.

jpgvm 6y ago
$DAY_JOB replaced helm with custom tool that does equivalent of kubectl apply by uses Jsonnet as templating engine. I intend to write an OSS version of it because Jsonnet really is probably the best fit here (though Cue/Dhall are interesting I think Jsonnet is easiest to get others to adopt).
- gravypod 6y ago
  I've done the same in the past. The important thing is making it so it's easy to build up abstractions and to work in a sensible deployment flow. kubectl is close but doesn't understand secrets/variables, the concept of clusters, or changing image tags programmatically. It's difficult to get right.
  - jpgvm 6y ago
    Yeah this is all stuff this tool understood, multiple clusters, managing kube contexts, external secret storage or k8s secrets, native git integration, etc. Definitely agree that it's difficult to get right but the authors of the tool definitely struck a really good balance. Hopefully if I implement it myself I can do their original design and implementation justice.
  - mdaniel 6y ago
    > kubectl is close but doesn't understand secrets/variables, the concept of clusters, or changing image tags programmatically. It's difficult to get right. Isn't that the problem kustomize (which kubectl now has native support for applying) is trying to solve? I ask that honestly, because I haven't used kustomize in anger in order to compare its workflow to that of helm
    - gravypod 6y ago
      My mistake. I meant kubecfg which is built off of jsonnet.
    - jpgvm 6y ago
      I did the full gambit of helm 1/2 tiller/tiller-less and kustomize. Personally kustomize is only marginally better than helm. I need to check out kubecfg, I haven't used it yet.
- etse 6y ago
  I also did something like this a few years ago (with Go templating instead of Jsonnet), but we recently replaced it with Kustomize (https://kustomize.io/ https://kustomize.io/) as it worked quite well for the basic use case of getting shared Kubernetes configs to deploy code to different environments at different times.
  - atmosx 6y ago
    The fact that trivial things like ingresses don’t have transformers is a PITA.
- qbasic_forever 6y ago
  No need to build it, there's already kubecfg: https://github.com/bitnami/kubecfg https://github.com/bitnami/kubecfg
  - rickspencer3 6y ago
    We use this heavily at InfluxData to manage multiple clusters in multiple regions of multiple clouds. One of the maintainers of kubecfg recently joined InfluxData as well.
qbasic_forever 6y ago
Helm isn't that connected to the broader Kubernetes system. It was popular in the early days when Kubernetes knowledge was sparse and things were still being figured out. None of the official Kubernetes docs really mention or describe using helm, and I see more getting started tutorials are starting to avoid it too. The official kubectl tool supports a -k mode which applies Kustomize logic and I'd argue Kustomize is probably the official way forward with app deployments by the Kubernetes project.
- KptMarchewa 6y ago
  Yet, the embedded kustomize version is years behind standalone. https://github.com/kubernetes-sigs/kustomize/issues/1500 https://github.com/kubernetes-sigs/kustomize/issues/1500

jpgvm 6y ago

$DAY_JOB replaced helm with custom tool that does equivalent of kubectl apply by uses Jsonnet as templating engine. I intend to write an OSS version of it because Jsonnet really is probably the best fit here (though Cue/Dhall are interesting I think Jsonnet is easiest to get others to adopt).

gravypod 6y ago
I've done the same in the past. The important thing is making it so it's easy to build up abstractions and to work in a sensible deployment flow. kubectl is close but doesn't understand secrets/variables, the concept of clusters, or changing image tags programmatically. It's difficult to get right.
- jpgvm 6y ago
  Yeah this is all stuff this tool understood, multiple clusters, managing kube contexts, external secret storage or k8s secrets, native git integration, etc. Definitely agree that it's difficult to get right but the authors of the tool definitely struck a really good balance. Hopefully if I implement it myself I can do their original design and implementation justice.
- mdaniel 6y ago
  > kubectl is close but doesn't understand secrets/variables, the concept of clusters, or changing image tags programmatically. It's difficult to get right. Isn't that the problem kustomize (which kubectl now has native support for applying) is trying to solve? I ask that honestly, because I haven't used kustomize in anger in order to compare its workflow to that of helm
  - gravypod 6y ago
    My mistake. I meant kubecfg which is built off of jsonnet.
  - jpgvm 6y ago
    I did the full gambit of helm 1/2 tiller/tiller-less and kustomize. Personally kustomize is only marginally better than helm. I need to check out kubecfg, I haven't used it yet.
etse 6y ago
I also did something like this a few years ago (with Go templating instead of Jsonnet), but we recently replaced it with Kustomize (https://kustomize.io/ https://kustomize.io/) as it worked quite well for the basic use case of getting shared Kubernetes configs to deploy code to different environments at different times.
- atmosx 6y ago
  The fact that trivial things like ingresses don’t have transformers is a PITA.
qbasic_forever 6y ago
No need to build it, there's already kubecfg: https://github.com/bitnami/kubecfg https://github.com/bitnami/kubecfg
- rickspencer3 6y ago
  We use this heavily at InfluxData to manage multiple clusters in multiple regions of multiple clouds. One of the maintainers of kubecfg recently joined InfluxData as well.

qbasic_forever 6y ago

Helm isn't that connected to the broader Kubernetes system. It was popular in the early days when Kubernetes knowledge was sparse and things were still being figured out. None of the official Kubernetes docs really mention or describe using helm, and I see more getting started tutorials are starting to avoid it too. The official kubectl tool supports a -k mode which applies Kustomize logic and I'd argue Kustomize is probably the official way forward with app deployments by the Kubernetes project.

KptMarchewa 6y ago
Yet, the embedded kustomize version is years behind standalone. https://github.com/kubernetes-sigs/kustomize/issues/1500 https://github.com/kubernetes-sigs/kustomize/issues/1500