5 ms·

Helm was something that I ultimately decided to avoid at my last job, and I was happier for it. Upgrades that get in dirty states, often as a result of bogus po

Helm was something that I ultimately decided to avoid at my last job, and I was happier for it. Upgrades that get in dirty states, often as a result of bogus post-install hooks. Having to fork charts to customize the deployment minorly for our mundane requirements. Reading doubly or triply templated files and dealing with bullshit indentation mismatches. I hated things that expected me to install with Helm.

nrmitchi 6y ago
I can agree with you that some of uses of it get out of hand. Like most things, there is a balance. For the most part I don't use hooks, precisely because they muddy the waters of my expected end state. Upgrades that get in bad states, ya, I've had to deal with it before, and it definitely sucks. The "atomic" goal that Helm 3 tries for is better in my opinion, but still has rough edges. What have you been using as an alternative that doesn't require you to write your own manifests for every dependency?
- blandflakes 6y ago
  A lot of the time, writing (or forking and customizing) my own manifest was at least more deterministic and predictable. Obviously you get lost at that point in time if you aren't diligent about upgrading. Helm felt like this middleground, and I really felt that at least for managed clusters, it would be better if the addon managers provided stuff that Just Worked (tm). We found a lot of success also with Kustomize. I did not enjoy the ksonnet detour as much - it was nice to override a prototype instead of having to screw with yaml templating, but jsonnet is not a good language. Helm 3 had some designs on a lua API, which I thought was a promising idea. I've been out of the loop since moving on to a company that is back in EC2-land (which really just means that another team manages the thing that approximates Kubernetes...).
  - nrmitchi 6y ago
    Ya, I've gone the fork-and-customize route (but I typically just `helm fetch`-d into my repo), or pinned to the version I needed. I agree with you about an addon manager, but then something has to be the source of truth for that, and the "KubeApps" type apps were all using Helm under the hood anyways. I played around with Kustomize for a bit at one point, but honestly had a bit of difficulty understanding how it loaded and applied the different "layers". I've been looking more recently at it as a "make some quick customizations to this rendered helm chart" tool as of late.
core-questions 6y ago
>Having to fork charts to customize the deployment minorly for our mundane requirements. This seems to be a big problem with container orchestrators these days. I was deploying something to ECS and while 90% of the config could be accomplished with environment variables, there are still a few cases where it was expected that I would provide files. On a local machine with Docker and some config mgmt like Chef or Ansible, no problem - template out those files and volume mount them in. In ECS? No easy to way to do it without baking your own container or using EFS, which for one config option seemed like major fuckin' overkill. Ended up bastardizing the entrypoint with some judicious base64'd bullshit and got it going without either, but christ, how fucking hard would it be to just provide a native mechanism for templating config files out?
- blandflakes 6y ago
  Absolutely. For applications I built myself, I would often just roll the possible configs for us as files in the container, and you pick one with a single env var (realistically this was like "dev", "beta", "prod"). Not great, but meant I could avoid the clusterfuck that is specifying all my AWS configuration in env vars on a per deploy basis. ConfigMaps could sort of help with this, but I found what you found - having to base64 a config file as a field in a ConfigMap and make sure it gets mounted correctly just seems pretty baroque for configuring something with a file. And at the time, ConfigMap changes didn't even cause a redeploy, so your pods would for their lifetime keep on using a stale config!
  - core-questions 6y ago
    Yeah, and this is when working with first-party upstream Dockerized apps that have good entrypoints already designed to function as adapters between env var config and internal config files....

nrmitchi 6y ago

I can agree with you that some of uses of it get out of hand. Like most things, there is a balance. For the most part I don't use hooks, precisely because they muddy the waters of my expected end state. Upgrades that get in bad states, ya, I've had to deal with it before, and it definitely sucks. The "atomic" goal that Helm 3 tries for is better in my opinion, but still has rough edges. What have you been using as an alternative that doesn't require you to write your own manifests for every dependency?

blandflakes 6y ago
A lot of the time, writing (or forking and customizing) my own manifest was at least more deterministic and predictable. Obviously you get lost at that point in time if you aren't diligent about upgrading. Helm felt like this middleground, and I really felt that at least for managed clusters, it would be better if the addon managers provided stuff that Just Worked (tm). We found a lot of success also with Kustomize. I did not enjoy the ksonnet detour as much - it was nice to override a prototype instead of having to screw with yaml templating, but jsonnet is not a good language. Helm 3 had some designs on a lua API, which I thought was a promising idea. I've been out of the loop since moving on to a company that is back in EC2-land (which really just means that another team manages the thing that approximates Kubernetes...).
- nrmitchi 6y ago
  Ya, I've gone the fork-and-customize route (but I typically just `helm fetch`-d into my repo), or pinned to the version I needed. I agree with you about an addon manager, but then something has to be the source of truth for that, and the "KubeApps" type apps were all using Helm under the hood anyways. I played around with Kustomize for a bit at one point, but honestly had a bit of difficulty understanding how it loaded and applied the different "layers". I've been looking more recently at it as a "make some quick customizations to this rendered helm chart" tool as of late.

core-questions 6y ago

>Having to fork charts to customize the deployment minorly for our mundane requirements. This seems to be a big problem with container orchestrators these days. I was deploying something to ECS and while 90% of the config could be accomplished with environment variables, there are still a few cases where it was expected that I would provide files. On a local machine with Docker and some config mgmt like Chef or Ansible, no problem - template out those files and volume mount them in. In ECS? No easy to way to do it without baking your own container or using EFS, which for one config option seemed like major fuckin' overkill. Ended up bastardizing the entrypoint with some judicious base64'd bullshit and got it going without either, but christ, how fucking hard would it be to just provide a native mechanism for templating config files out?

blandflakes 6y ago
Absolutely. For applications I built myself, I would often just roll the possible configs for us as files in the container, and you pick one with a single env var (realistically this was like "dev", "beta", "prod"). Not great, but meant I could avoid the clusterfuck that is specifying all my AWS configuration in env vars on a per deploy basis. ConfigMaps could sort of help with this, but I found what you found - having to base64 a config file as a field in a ConfigMap and make sure it gets mounted correctly just seems pretty baroque for configuring something with a file. And at the time, ConfigMap changes didn't even cause a redeploy, so your pods would for their lifetime keep on using a stale config!
- core-questions 6y ago
  Yeah, and this is when working with first-party upstream Dockerized apps that have good entrypoints already designed to function as adapters between env var config and internal config files....