5 ms·

I 100% see the point you are trying to make. However Kubernetes does have a lot to offer, especially with the easy of hosted Kubernetes solutions these days. H

I 100% see the point you are trying to make. However Kubernetes does have a lot to offer, especially with the easy of hosted Kubernetes solutions these days. Helm makes installing tools and dependencies you may need almost trivial. One of the nice things about Kubernetes is that it is relatively easy to scale from a small deployment to a larger HA deployment when (and if) you need to. > Last year I helped a client who had raised $5 million and then, before they hired me, they'd wasted $1 million on a "high availability" devops setup that was overbuilt for what they needed. This sounds like more of a management, requirements, and overbuilding problem, rather than a Kubernetes problem. Even without Kubernetes, if they wanted an HA devops set up, they still would have built one. Honestly without Kubernetes it could have ended up more expensive.

blandflakes 6y ago
Helm was something that I ultimately decided to avoid at my last job, and I was happier for it. Upgrades that get in dirty states, often as a result of bogus post-install hooks. Having to fork charts to customize the deployment minorly for our mundane requirements. Reading doubly or triply templated files and dealing with bullshit indentation mismatches. I hated things that expected me to install with Helm.
- nrmitchi 6y ago
  I can agree with you that some of uses of it get out of hand. Like most things, there is a balance. For the most part I don't use hooks, precisely because they muddy the waters of my expected end state. Upgrades that get in bad states, ya, I've had to deal with it before, and it definitely sucks. The "atomic" goal that Helm 3 tries for is better in my opinion, but still has rough edges. What have you been using as an alternative that doesn't require you to write your own manifests for every dependency?
  - blandflakes 6y ago
    A lot of the time, writing (or forking and customizing) my own manifest was at least more deterministic and predictable. Obviously you get lost at that point in time if you aren't diligent about upgrading. Helm felt like this middleground, and I really felt that at least for managed clusters, it would be better if the addon managers provided stuff that Just Worked (tm). We found a lot of success also with Kustomize. I did not enjoy the ksonnet detour as much - it was nice to override a prototype instead of having to screw with yaml templating, but jsonnet is not a good language. Helm 3 had some designs on a lua API, which I thought was a promising idea. I've been out of the loop since moving on to a company that is back in EC2-land (which really just means that another team manages the thing that approximates Kubernetes...).
    - nrmitchi 6y ago
      Ya, I've gone the fork-and-customize route (but I typically just `helm fetch`-d into my repo), or pinned to the version I needed. I agree with you about an addon manager, but then something has to be the source of truth for that, and the "KubeApps" type apps were all using Helm under the hood anyways. I played around with Kustomize for a bit at one point, but honestly had a bit of difficulty understanding how it loaded and applied the different "layers". I've been looking more recently at it as a "make some quick customizations to this rendered helm chart" tool as of late.
- core-questions 6y ago
  >Having to fork charts to customize the deployment minorly for our mundane requirements. This seems to be a big problem with container orchestrators these days. I was deploying something to ECS and while 90% of the config could be accomplished with environment variables, there are still a few cases where it was expected that I would provide files. On a local machine with Docker and some config mgmt like Chef or Ansible, no problem - template out those files and volume mount them in. In ECS? No easy to way to do it without baking your own container or using EFS, which for one config option seemed like major fuckin' overkill. Ended up bastardizing the entrypoint with some judicious base64'd bullshit and got it going without either, but christ, how fucking hard would it be to just provide a native mechanism for templating config files out?
  - blandflakes 6y ago
    Absolutely. For applications I built myself, I would often just roll the possible configs for us as files in the container, and you pick one with a single env var (realistically this was like "dev", "beta", "prod"). Not great, but meant I could avoid the clusterfuck that is specifying all my AWS configuration in env vars on a per deploy basis. ConfigMaps could sort of help with this, but I found what you found - having to base64 a config file as a field in a ConfigMap and make sure it gets mounted correctly just seems pretty baroque for configuring something with a file. And at the time, ConfigMap changes didn't even cause a redeploy, so your pods would for their lifetime keep on using a stale config!
    - core-questions 6y ago
      Yeah, and this is when working with first-party upstream Dockerized apps that have good entrypoints already designed to function as adapters between env var config and internal config files....

blandflakes 6y ago

Helm was something that I ultimately decided to avoid at my last job, and I was happier for it. Upgrades that get in dirty states, often as a result of bogus post-install hooks. Having to fork charts to customize the deployment minorly for our mundane requirements. Reading doubly or triply templated files and dealing with bullshit indentation mismatches. I hated things that expected me to install with Helm.

nrmitchi 6y ago
I can agree with you that some of uses of it get out of hand. Like most things, there is a balance. For the most part I don't use hooks, precisely because they muddy the waters of my expected end state. Upgrades that get in bad states, ya, I've had to deal with it before, and it definitely sucks. The "atomic" goal that Helm 3 tries for is better in my opinion, but still has rough edges. What have you been using as an alternative that doesn't require you to write your own manifests for every dependency?
- blandflakes 6y ago
  A lot of the time, writing (or forking and customizing) my own manifest was at least more deterministic and predictable. Obviously you get lost at that point in time if you aren't diligent about upgrading. Helm felt like this middleground, and I really felt that at least for managed clusters, it would be better if the addon managers provided stuff that Just Worked (tm). We found a lot of success also with Kustomize. I did not enjoy the ksonnet detour as much - it was nice to override a prototype instead of having to screw with yaml templating, but jsonnet is not a good language. Helm 3 had some designs on a lua API, which I thought was a promising idea. I've been out of the loop since moving on to a company that is back in EC2-land (which really just means that another team manages the thing that approximates Kubernetes...).
  - nrmitchi 6y ago
    Ya, I've gone the fork-and-customize route (but I typically just `helm fetch`-d into my repo), or pinned to the version I needed. I agree with you about an addon manager, but then something has to be the source of truth for that, and the "KubeApps" type apps were all using Helm under the hood anyways. I played around with Kustomize for a bit at one point, but honestly had a bit of difficulty understanding how it loaded and applied the different "layers". I've been looking more recently at it as a "make some quick customizations to this rendered helm chart" tool as of late.
core-questions 6y ago
>Having to fork charts to customize the deployment minorly for our mundane requirements. This seems to be a big problem with container orchestrators these days. I was deploying something to ECS and while 90% of the config could be accomplished with environment variables, there are still a few cases where it was expected that I would provide files. On a local machine with Docker and some config mgmt like Chef or Ansible, no problem - template out those files and volume mount them in. In ECS? No easy to way to do it without baking your own container or using EFS, which for one config option seemed like major fuckin' overkill. Ended up bastardizing the entrypoint with some judicious base64'd bullshit and got it going without either, but christ, how fucking hard would it be to just provide a native mechanism for templating config files out?
- blandflakes 6y ago
  Absolutely. For applications I built myself, I would often just roll the possible configs for us as files in the container, and you pick one with a single env var (realistically this was like "dev", "beta", "prod"). Not great, but meant I could avoid the clusterfuck that is specifying all my AWS configuration in env vars on a per deploy basis. ConfigMaps could sort of help with this, but I found what you found - having to base64 a config file as a field in a ConfigMap and make sure it gets mounted correctly just seems pretty baroque for configuring something with a file. And at the time, ConfigMap changes didn't even cause a redeploy, so your pods would for their lifetime keep on using a stale config!
  - core-questions 6y ago
    Yeah, and this is when working with first-party upstream Dockerized apps that have good entrypoints already designed to function as adapters between env var config and internal config files....