4 ms·

Helm is horrible. Please use Kustomize. Life will get better, it did for me. I also use Sops (with goabout/kustomize-sopssecretgenerator) and ArgoCD for a very

Helm is horrible. Please use Kustomize. Life will get better, it did for me. I also use Sops (with goabout/kustomize-sopssecretgenerator) and ArgoCD for a very slick well integrated system. The CRDs in ArgoCD are so well done that it feels like native K8s CD.

booleanbetrayal 5y ago
Big fan of ArgoCD here. The development team is also very responsive with good processes in place, from my experience in the repo.
mfer 5y ago
Kustomize and Helm take different routes. Kustomize expects the user to know Kubernetes resources and their schemas. Helm and charts target the chart creator being being an expert in Kubernetes while the chart consumer (the Helm user) doesn't need to be an expert. They can have a simple interface to deploy an application. One of the issues for those running applications in Kubernetes is the complexity. Kubernetes is complex. So much so that people like Kelsey Hightower have said Kubernetes is a platform for creating platforms. For most people to learn Kubernetes will slow down their velocity which is a negative. Most people need a simpler system on top of Kubernetes to make them effective.
- sascha_sl 5y ago
  Helm itself is pretty awful through. For instance, when people upgraded to 1.16, where Kubernetes retired apps/v1beta1, Kubernetes automatically transparently converts these types. Helm, having once created these resources as v1beta1 will completely refuse to touch the deployment. Without some specialized extra tooling that people created in the wake of this, resolving this requires deleting the entire namespace and reinstalling the chart. Issue on helm repo closed as wontfix.
  - mfer 5y ago
    This still comes down to assumptions. If someone was using raw kubernetes manifests instead of Helm and charts they would need to know the schemas and details of those manifests. Then, when 1.16 came out they would need to update to the new apiVersion and modify their manifests for any differences in schema. This requires someone who knows and understands k8s resources. This IS NOT a typical app dev or person who wants to run apps in k8s. I've been in numerous circles of app folks who have complained about this expectation from the k8s community. On the main Helm project they have the stance that what's in a chart is up to the charts authors. Just like what's in a debian package is what's up to the debian package authors. That those folks should do a good job maintaining them. That's why it was a wontfix issue. What we see as an underlying issue with k8s is the experience that app devs and people who want to run apps have to deal with. It's currently like expecting an app dev to write in assembly and know when assembly codes are deprecated or changed. It's in the pre-high level language phase. This comparison to assembly is one that's come out of the k8s community itself.
    - sascha_sl 5y ago
      I generally do not disagree that the experience for appdevs on k8s is bad, but I also disagree that they should be the audience to begin with. Helm is still functionally broken. A helm chart that was kept up to date still could've used v1beta1 early on. All it takes is a cluster that hasn't been patched in a while. This is 100% a helm issue. (Note that this bug also triggers if the chart has been updated, but helm recorded a different GVK) What you actually need is a platform team that provides a middle layer. I was part of such a team between 2018 and 2021, and my main takeaway was to not let developers have write access at all after slicing a namespace with 25k pods out of etcd because gRPC refused to operate on it at that point (that was a misconfigured CronJob). You need something like KubeVela, not Helm. https://kubevela.io/ https://kubevela.io/
- joshuak 5y ago
  There is no need to use Kubernetes if it slows down your velocity, just use docker or direct drive some VMs. If you NEED Kubernetes then you simply won't get most of the benefits it provides by learning some short cut tools. Tools like helm are easy to use, but do so at the cost of breaking core design elements and a massive increase in complexity just below the surface. Here's the rule of thumb for Kubernetes use tools to manage the files not the cluster, the cluster description is the source of truth not the cluster.

booleanbetrayal 5y ago

Big fan of ArgoCD here. The development team is also very responsive with good processes in place, from my experience in the repo.

mfer 5y ago

Kustomize and Helm take different routes. Kustomize expects the user to know Kubernetes resources and their schemas. Helm and charts target the chart creator being being an expert in Kubernetes while the chart consumer (the Helm user) doesn't need to be an expert. They can have a simple interface to deploy an application. One of the issues for those running applications in Kubernetes is the complexity. Kubernetes is complex. So much so that people like Kelsey Hightower have said Kubernetes is a platform for creating platforms. For most people to learn Kubernetes will slow down their velocity which is a negative. Most people need a simpler system on top of Kubernetes to make them effective.

sascha_sl 5y ago
Helm itself is pretty awful through. For instance, when people upgraded to 1.16, where Kubernetes retired apps/v1beta1, Kubernetes automatically transparently converts these types. Helm, having once created these resources as v1beta1 will completely refuse to touch the deployment. Without some specialized extra tooling that people created in the wake of this, resolving this requires deleting the entire namespace and reinstalling the chart. Issue on helm repo closed as wontfix.
- mfer 5y ago
  This still comes down to assumptions. If someone was using raw kubernetes manifests instead of Helm and charts they would need to know the schemas and details of those manifests. Then, when 1.16 came out they would need to update to the new apiVersion and modify their manifests for any differences in schema. This requires someone who knows and understands k8s resources. This IS NOT a typical app dev or person who wants to run apps in k8s. I've been in numerous circles of app folks who have complained about this expectation from the k8s community. On the main Helm project they have the stance that what's in a chart is up to the charts authors. Just like what's in a debian package is what's up to the debian package authors. That those folks should do a good job maintaining them. That's why it was a wontfix issue. What we see as an underlying issue with k8s is the experience that app devs and people who want to run apps have to deal with. It's currently like expecting an app dev to write in assembly and know when assembly codes are deprecated or changed. It's in the pre-high level language phase. This comparison to assembly is one that's come out of the k8s community itself.
  - sascha_sl 5y ago
    I generally do not disagree that the experience for appdevs on k8s is bad, but I also disagree that they should be the audience to begin with. Helm is still functionally broken. A helm chart that was kept up to date still could've used v1beta1 early on. All it takes is a cluster that hasn't been patched in a while. This is 100% a helm issue. (Note that this bug also triggers if the chart has been updated, but helm recorded a different GVK) What you actually need is a platform team that provides a middle layer. I was part of such a team between 2018 and 2021, and my main takeaway was to not let developers have write access at all after slicing a namespace with 25k pods out of etcd because gRPC refused to operate on it at that point (that was a misconfigured CronJob). You need something like KubeVela, not Helm. https://kubevela.io/ https://kubevela.io/
joshuak 5y ago
There is no need to use Kubernetes if it slows down your velocity, just use docker or direct drive some VMs. If you NEED Kubernetes then you simply won't get most of the benefits it provides by learning some short cut tools. Tools like helm are easy to use, but do so at the cost of breaking core design elements and a massive increase in complexity just below the surface. Here's the rule of thumb for Kubernetes use tools to manage the files not the cluster, the cluster description is the source of truth not the cluster.