4 ms·

This is a big step forward for Helm. Tiller has been a major pain point in the past, although I understand the design decisions why it was placed here in the pa

by cfors 7y ago

This is a big step forward for Helm. Tiller has been a major pain point in the past, although I understand the design decisions why it was placed here in the past. > In Helm 3, we now use a three-way strategic merge patch. Helm considers the old manifest, its live state, and the new manifest when generating a patch. I will have to test this out, but honestly I think that any sort of "strategic" merge is probably the wrong approach to take when dealing with so called "idempotent" infrastructure.

ses1984 7y ago
Strategy is how you deal with idempotency and consistency in a distributed environment. Your manifest is idempotent but isn't applied instantly. You can apply a manifest, and get into some broken state, how do you recover? What if you try to roll back and things get worse?
- cfors 7y ago
  I suppose if you're not using git-ops related deployments, but Git has been proven to work well for this exact process. If you are tracking your releases as part of a Git commit, then you rollback with that. Doesn't seem that complicated, but maybe I'm missing some benefits of this strategy that others with more experience managing K8s resources have had. Edit: Retroactively speaking you also don't need to leverage etcd to store n*x your K8s resources in a secrets file where n is the potential rollback number, it's all stored in a Git repository.
the_duke 7y ago
Tiller has indeed been a major pain point. Do you know how v3 prevents race conditions though? I assume there is some sort of locking mechanism via custom resources?
- fernandotakai 7y ago
  tiller was, imho, a huge burden on even testing out helm. setting up the right permissions was such a pain in the ass.
  - mfer 7y ago
    To add some context... Tiller was not part of Helm v1. Tiller came in when Helm (which was developed outside of the Kubernetes project) was merged with deployment manager (which was part of Kubernetes). This produced Helm v2 which was part of Kubernetes. Helm grew large enough to spin off into it's own CNCF project. Tiller was created early enough in the project that Secrets didn't exist at the time. This was long before RBAC or even the workloads API. Tiller could have been remade as a custom controller which is how many things are created today. But, many people who share their apps work at organizations that don't let them install CRDs or use models like that. So, keeping the Helm footprint to a level that let's them and most others use it, if they choose, was a reason to remove Tiller. It also simplified the codebase a great deal.
mfer 7y ago
To add context, when updating objects in place a strategic merge is how the Kubernetes docs tell you to do and they they provide a kubectl example [1]. If this is not the right way to do it I would suggest that conversation would be good to have in the Kubernetes project itself. [1] https://kubernetes.io/docs/tasks/run-application/update-api-object-kubectl-patch/ https://kubernetes.io/docs/tasks/run-application/update-api-...
- cfors 7y ago
  I am aware of that, and here is one the of the issues I have with the strategic patch strategy: > The patch you did in the preceding exercise is called a strategic merge patch. Notice that the patch did not replace the containers list. Instead it added a new Container to the list. In other words, the list in the patch was merged with the existing list. This is not always what happens when you use a strategic merge patch on a list. In some cases, the list is replaced, not merged. [0] This seems like a lot of cognitive overload. I understand there are some use cases for this but really, all I want is to have my K8s resources all tracked with a git commit reference and then that is what is deployed exactly. [0] https://kubernetes.io/docs/tasks/run-application/update-api-object-kubectl-patch/#notes-on-the-strategic-merge-patch https://kubernetes.io/docs/tasks/run-application/update-api-...
  - bacongobbler 7y ago
    This is not how things work in reality. Many last-mile objects are merged into your Kubernetes resources at the last minute. Service meshes inject sidecar containers into your deployments. The Kubernetes API can update a Service's virtual IP addresses which can change over time. All of these object updates need to be taken into consideration during an upgrade, or you risk disrupting resources running in production.
    - cfors 7y ago
      I'm not saying there isn't a use case for this, it's just that for the most part I prefer dumping what I have from an immutable git commit into the Kube API. If something needs to happen after the fact for governance or injecting sidecars, so be it but that is out of the scope of what I am deploying.

by cfors 7y ago

ses1984 7y ago

Strategy is how you deal with idempotency and consistency in a distributed environment. Your manifest is idempotent but isn't applied instantly. You can apply a manifest, and get into some broken state, how do you recover? What if you try to roll back and things get worse?

cfors 7y ago
I suppose if you're not using git-ops related deployments, but Git has been proven to work well for this exact process. If you are tracking your releases as part of a Git commit, then you rollback with that. Doesn't seem that complicated, but maybe I'm missing some benefits of this strategy that others with more experience managing K8s resources have had. Edit: Retroactively speaking you also don't need to leverage etcd to store n*x your K8s resources in a secrets file where n is the potential rollback number, it's all stored in a Git repository.

the_duke 7y ago

Tiller has indeed been a major pain point. Do you know how v3 prevents race conditions though? I assume there is some sort of locking mechanism via custom resources?

fernandotakai 7y ago
tiller was, imho, a huge burden on even testing out helm. setting up the right permissions was such a pain in the ass.
- mfer 7y ago
  To add some context... Tiller was not part of Helm v1. Tiller came in when Helm (which was developed outside of the Kubernetes project) was merged with deployment manager (which was part of Kubernetes). This produced Helm v2 which was part of Kubernetes. Helm grew large enough to spin off into it's own CNCF project. Tiller was created early enough in the project that Secrets didn't exist at the time. This was long before RBAC or even the workloads API. Tiller could have been remade as a custom controller which is how many things are created today. But, many people who share their apps work at organizations that don't let them install CRDs or use models like that. So, keeping the Helm footprint to a level that let's them and most others use it, if they choose, was a reason to remove Tiller. It also simplified the codebase a great deal.

mfer 7y ago

To add context, when updating objects in place a strategic merge is how the Kubernetes docs tell you to do and they they provide a kubectl example [1]. If this is not the right way to do it I would suggest that conversation would be good to have in the Kubernetes project itself. [1] https://kubernetes.io/docs/tasks/run-application/update-api-object-kubectl-patch/ https://kubernetes.io/docs/tasks/run-application/update-api-...

cfors 7y ago
I am aware of that, and here is one the of the issues I have with the strategic patch strategy: > The patch you did in the preceding exercise is called a strategic merge patch. Notice that the patch did not replace the containers list. Instead it added a new Container to the list. In other words, the list in the patch was merged with the existing list. This is not always what happens when you use a strategic merge patch on a list. In some cases, the list is replaced, not merged. [0] This seems like a lot of cognitive overload. I understand there are some use cases for this but really, all I want is to have my K8s resources all tracked with a git commit reference and then that is what is deployed exactly. [0] https://kubernetes.io/docs/tasks/run-application/update-api-object-kubectl-patch/#notes-on-the-strategic-merge-patch https://kubernetes.io/docs/tasks/run-application/update-api-...
- bacongobbler 7y ago
  This is not how things work in reality. Many last-mile objects are merged into your Kubernetes resources at the last minute. Service meshes inject sidecar containers into your deployments. The Kubernetes API can update a Service's virtual IP addresses which can change over time. All of these object updates need to be taken into consideration during an upgrade, or you risk disrupting resources running in production.
  - cfors 7y ago
    I'm not saying there isn't a use case for this, it's just that for the most part I prefer dumping what I have from an immutable git commit into the Kube API. If something needs to happen after the fact for governance or injecting sidecars, so be it but that is out of the scope of what I am deploying.