7 ms·

Each of those benefits are things I had before using containers or kubernetes, and were simpler. > Builds with fixed dependencies that never change. Rollback i

Each of those benefits are things I had before using containers or kubernetes, and were simpler. > Builds with fixed dependencies that never change. Rollback is easy Any good build system already did this, such as Bazel, or a Gemfile.lock. We'd just snapshot AMIs to keep OS dependencies fixed... which is what Docker images effectively do. If you re-docker-build the same Dockerfile, it's not like you get the same result of "apt-get install libxml" the next time either. > Easy deployment of a prod environment on a local machine How containers are deployed varies wildly between prod and the local machine. All the things that were hard before are still hard. Things like secrets and external dependencies still usually vary. If prod is a kubernetes environment, getting a suitable k8s environment setup locally sucks, especially since it will probably have a different ingress controller, load balancer setup, storage classes available, resource requests, etc. If prod is kubernetes and local is docker-compose, that honestly seems like just as much work to create a second way to run the stack than just using a bash script + "npm start" or "bundle exec rails server" or whatever. Either way, it's not really a prod environment. It's hard to run identical-to-prod environments locally, and those problems are related to secrets and clouds and such, not due to the lack of containers, in my experience. > Fast deployment In my experience, containers haven't sped up deployment. Let's say you use ubuntu for your host and container's OS. Before containers, this meant you had to download one version of libssl ever, and that was it. If there was an update to libz, that didn't require a new download of libssl. After containers, if you build your container for app1 last week, and your container for app2 today, the "FROM ubuntu" likely resolves to a different image. Both your apps now have different "ubuntu" layers, which probably have the same version of libssl, but deduplication of downloads only happens if the whole layer is identical. In essence, we went from downloading 1 copy of libssl (for the host OS only) to 3 copies (host OS + 2 containers w/ different ubuntu bases), and there's no deduplication. That by itself seems like it has to be slower since there's an inherent increase in network bandwidth that has to happen. Even if you have a shared base image, you're at least doubling the downloads of libssl since before you could use the host's copy only. All the items you listed under k8s are things I had before it, excluding "Abstraction of infrastructure". Frankly, if you have a well-made load balancer, it's hard not to have zero-downtime deployments and auto-scaling.

mixermachine 5y ago
> We'd just snapshot AMIs to keep OS dependencies fixed This is a good solution, but I would not call it easier. Using docker container feels like installing an app on my smartphone. I choose the version and it will always work like I build it at date x without an additional system. Works for every programming language with every dependency out of the box. Python, Java, Javascript, GO, Ocaml, C, ... > How containers are deployed varies wildly between prod and the local machine I just brought a product of my company to Kubernetes. Run helm upgrade --install . -f dev-values.yaml for dev Run helm upgrade --install . -f prod-values.yaml for prod (of course you need the secrets there. Jenkins has them). My laptop does run an environment with all components of the prod env. Something like email and sap services are of course mocked, but everything else? All on my machine. Why not? I can spin up a new test environment for customers with new settings on the same day. > Both your apps now have different "ubuntu" layers We use a base image that does change not that often. Even if: no problem, the registry is connected via 1000 MBit/s and zero-downtime deployment does its magic so I don't even notice if it takes one or two minutes. Another thing: my node (or VM) libs and the libs of my software should not be connected in any way (at least for me). I want to patch my nodes and my software independently. Different software should also not be bound to libs of another software. > All the items you listed under k8s are things I had before it - How do you easily scale up? Including starting new machines and spinning down machines that are no longer needed - How are multiple software parts executed on one host? - How do you do fail over? I know that everything can be done without Kubernetes. With enough time and money one can create large systems that do this. I spun up a new Kubernetes cluster and ported our product (already containerized) on the cluster in about three months. Really: I also love the classic dev ops and have a proxmox server at home, but Kubernetes just solves many problems at once in a short time.
- TheDong 5y ago
  > How do you easily scale up? Including starting new machines and spinning down machines that are no longer needed AWS autoscaling groups + cloudwatch for adding and removing machines + checking them into load balancers is something that has worked for longer than K8s has been a thing. > How are multiple software parts executed on one host? systemd units, or for more resource hungry things, multiple autoscaling groups. The overhead of running the kubelet on each host + etcd cluster + apiserver means that I still end up with fewer hosts if I just run each component on every single host vs scaling different deployments independently. It is true that kubernetes might be more resource efficient in some combination of nodes and software, but at under 10 servers, I've always found the overhead of the etcd cluster + apiserver + kubelet to dwarf any savings from not just running 10 copies of my software. > How do you do fail over? The AWS-managed load balancer can fail over based on health checks failing, metrics, or I can add/remove servers from it manually. You can also do DNS health checks, or add a layer of haproxy/nginx/whatever if you want. It's not like k8s has some magic ability to fail over under the hood. It's just using k8s service objects (probably LoadBalancer type), which does the same thing.

mixermachine 5y ago

> We'd just snapshot AMIs to keep OS dependencies fixed This is a good solution, but I would not call it easier. Using docker container feels like installing an app on my smartphone. I choose the version and it will always work like I build it at date x without an additional system. Works for every programming language with every dependency out of the box. Python, Java, Javascript, GO, Ocaml, C, ... > How containers are deployed varies wildly between prod and the local machine I just brought a product of my company to Kubernetes. Run helm upgrade --install . -f dev-values.yaml for dev Run helm upgrade --install . -f prod-values.yaml for prod (of course you need the secrets there. Jenkins has them). My laptop does run an environment with all components of the prod env. Something like email and sap services are of course mocked, but everything else? All on my machine. Why not? I can spin up a new test environment for customers with new settings on the same day. > Both your apps now have different "ubuntu" layers We use a base image that does change not that often. Even if: no problem, the registry is connected via 1000 MBit/s and zero-downtime deployment does its magic so I don't even notice if it takes one or two minutes. Another thing: my node (or VM) libs and the libs of my software should not be connected in any way (at least for me). I want to patch my nodes and my software independently. Different software should also not be bound to libs of another software. > All the items you listed under k8s are things I had before it - How do you easily scale up? Including starting new machines and spinning down machines that are no longer needed - How are multiple software parts executed on one host? - How do you do fail over? I know that everything can be done without Kubernetes. With enough time and money one can create large systems that do this. I spun up a new Kubernetes cluster and ported our product (already containerized) on the cluster in about three months. Really: I also love the classic dev ops and have a proxmox server at home, but Kubernetes just solves many problems at once in a short time.

TheDong 5y ago
> How do you easily scale up? Including starting new machines and spinning down machines that are no longer needed AWS autoscaling groups + cloudwatch for adding and removing machines + checking them into load balancers is something that has worked for longer than K8s has been a thing. > How are multiple software parts executed on one host? systemd units, or for more resource hungry things, multiple autoscaling groups. The overhead of running the kubelet on each host + etcd cluster + apiserver means that I still end up with fewer hosts if I just run each component on every single host vs scaling different deployments independently. It is true that kubernetes might be more resource efficient in some combination of nodes and software, but at under 10 servers, I've always found the overhead of the etcd cluster + apiserver + kubelet to dwarf any savings from not just running 10 copies of my software. > How do you do fail over? The AWS-managed load balancer can fail over based on health checks failing, metrics, or I can add/remove servers from it manually. You can also do DNS health checks, or add a layer of haproxy/nginx/whatever if you want. It's not like k8s has some magic ability to fail over under the hood. It's just using k8s service objects (probably LoadBalancer type), which does the same thing.