4 ms·

Thanks for the detailed response. It looks like I've still got a lot to learn - I've just lately been playing with LXC to get more familiarised with containers.

by Ianvdl 6y ago

Thanks for the detailed response. It looks like I've still got a lot to learn - I've just lately been playing with LXC to get more familiarised with containers. I've previously looked at Helm apps and they seemed to be very similar to Puppet manifests. From what you said it seems like the approach is to have immutable containers for each application, set up via Dockerfiles, which somehow also simplifies the upgrade process? Does that mean you just deploy a new version/container of an application linked to the same underlying database (for example) when you need to run an upgrade? So if you had a fleet of ten containers running the same application in a load balanced config, I'm guessing you'd need to upgrade all of them at once (with downtime) rather than upgrading them one by one (because then the database would be inconsistent)? I'm assuming that since the containers are immutable the data is stored elsewhere.

lukeschlather 6y ago
In Kubernetes, deploying 10 containers takes a minute or two. I haven't worked with incredibly large deployments, but really deploying any amount of containers could easily take a minute or two if you have enough nodes. There is no downtime. Database inconsistency can cause problems but also any problems like that can be mitigated by doing a two-phase change to the database, and such changes are pretty rare and also devs instinctively avoid making those kinds of schema changes.
tsimionescu 6y ago
> So if you had a fleet of ten containers running the same application in a load balanced config, I'm guessing you'd need to upgrade all of them at once (with downtime) rather than upgrading them one by one (because then the database would be inconsistent)? That depends entirely on your application and the upgrade itself. Assuming we are discussing 10 different containers (e.g. 10 micro-services), k8s will normally update them in parallel but not atomically, it would be up to apication or deployment time logic to ensure they are updated as a single 'transaction'. If they are 10 copies of the same container, then k8s itself has tools for rolling upgrades where you can control the rollout. Also, depending on application logic, the upgrade could be done in such a way that there is no need to synchronize the services, they could work with the DB as is.
specialp 6y ago
You get the load balancing for free in K8s and rolling deploys. What you do is upgrade the deployment with a new docker image, and yes immutably it is replaced. In a case of an HTTP service, k8s will wait until a pod (container) responds healthy until it is put in the loop. Then it steps down old pods according to your rolling deploy metrics and replaces them. You can define what that is like having a max number of pods with a minimum number of pods up.
mbushey 6y ago
Helm is so bad it raises my blood pressure just hearing the name. Helm tries to apply the old way of doing things (like as you said Puppet) and makes it worse than ever. K8s yaml config is simple and elegant, don't try hiding it under templates. Kustomize is the proper way of working with K8s yaml. Helm fights against it.
- anchochilis 6y ago
  Helm isn't great (god, Go templates, shudder), but I love being able to bundle all the manifests for an application together with well-documented parameters/values... If you want do something like conditionally switch from a LoadBalancer service to Ingress in a test environment... I have no idea how you'd handle that in Kustomize, but it's straightforward in Helm. Ultimately it seems like you end up with the same complexity, just expressed via 100 layers of Kustomize transforms instead of with a conditional in your template.
  - mbushey 6y ago
    There are two different types of patches in Kustomize. It does take a bit to get used to it as it's different. jid (Interactive jq) makes the process a lot easier. It's actually pretty easy to make changes like this.

by Ianvdl 6y ago

lukeschlather 6y ago

In Kubernetes, deploying 10 containers takes a minute or two. I haven't worked with incredibly large deployments, but really deploying any amount of containers could easily take a minute or two if you have enough nodes. There is no downtime. Database inconsistency can cause problems but also any problems like that can be mitigated by doing a two-phase change to the database, and such changes are pretty rare and also devs instinctively avoid making those kinds of schema changes.

tsimionescu 6y ago

> So if you had a fleet of ten containers running the same application in a load balanced config, I'm guessing you'd need to upgrade all of them at once (with downtime) rather than upgrading them one by one (because then the database would be inconsistent)? That depends entirely on your application and the upgrade itself. Assuming we are discussing 10 different containers (e.g. 10 micro-services), k8s will normally update them in parallel but not atomically, it would be up to apication or deployment time logic to ensure they are updated as a single 'transaction'. If they are 10 copies of the same container, then k8s itself has tools for rolling upgrades where you can control the rollout. Also, depending on application logic, the upgrade could be done in such a way that there is no need to synchronize the services, they could work with the DB as is.

specialp 6y ago

You get the load balancing for free in K8s and rolling deploys. What you do is upgrade the deployment with a new docker image, and yes immutably it is replaced. In a case of an HTTP service, k8s will wait until a pod (container) responds healthy until it is put in the loop. Then it steps down old pods according to your rolling deploy metrics and replaces them. You can define what that is like having a max number of pods with a minimum number of pods up.

mbushey 6y ago

Helm is so bad it raises my blood pressure just hearing the name. Helm tries to apply the old way of doing things (like as you said Puppet) and makes it worse than ever. K8s yaml config is simple and elegant, don't try hiding it under templates. Kustomize is the proper way of working with K8s yaml. Helm fights against it.

anchochilis 6y ago
Helm isn't great (god, Go templates, shudder), but I love being able to bundle all the manifests for an application together with well-documented parameters/values... If you want do something like conditionally switch from a LoadBalancer service to Ingress in a test environment... I have no idea how you'd handle that in Kustomize, but it's straightforward in Helm. Ultimately it seems like you end up with the same complexity, just expressed via 100 layers of Kustomize transforms instead of with a conditional in your template.
- mbushey 6y ago
  There are two different types of patches in Kustomize. It does take a bit to get used to it as it's different. jid (Interactive jq) makes the process a lot easier. It's actually pretty easy to make changes like this.