7 ms·

Moving from classic servers to containers you get: - Builds with fixed dependencies that never change. Rollback is easy - Easy deployment of a prod environmen

Moving from classic servers to containers you get: - Builds with fixed dependencies that never change. Rollback is easy - Easy deployment of a prod environment on a local machine - Fast deployment - Easy automation (use version X with config Y) With Kubernetes (or other derivates like Openshift) you get: - Auto scaling - Fail over - Better resource usage if multiple environments are executed - Abstraction of infrastructure - Zero downtime deployment (biggest point for my company, we deploy >3 times per week) There are applications that do not need Kubernetes or even containers, but is this list really nothing oO? I can imagine that if you use Kubernetes just like a classic cluster it could seem like an unnecesarry added complexity but you gain a lot of things.

zz865 5y ago
Yeah we have fixed usage so scaling or easy failover is not something we need.
- mixermachine 5y ago
  Then I can understand you well. Kubernetes then just provides zero-downtime and additional complexity. When you already have something like a deployment window (like 2 am to 3 am) then ZDT also does not matter.
TheDong 5y ago
Each of those benefits are things I had before using containers or kubernetes, and were simpler. > Builds with fixed dependencies that never change. Rollback is easy Any good build system already did this, such as Bazel, or a Gemfile.lock. We'd just snapshot AMIs to keep OS dependencies fixed... which is what Docker images effectively do. If you re-docker-build the same Dockerfile, it's not like you get the same result of "apt-get install libxml" the next time either. > Easy deployment of a prod environment on a local machine How containers are deployed varies wildly between prod and the local machine. All the things that were hard before are still hard. Things like secrets and external dependencies still usually vary. If prod is a kubernetes environment, getting a suitable k8s environment setup locally sucks, especially since it will probably have a different ingress controller, load balancer setup, storage classes available, resource requests, etc. If prod is kubernetes and local is docker-compose, that honestly seems like just as much work to create a second way to run the stack than just using a bash script + "npm start" or "bundle exec rails server" or whatever. Either way, it's not really a prod environment. It's hard to run identical-to-prod environments locally, and those problems are related to secrets and clouds and such, not due to the lack of containers, in my experience. > Fast deployment In my experience, containers haven't sped up deployment. Let's say you use ubuntu for your host and container's OS. Before containers, this meant you had to download one version of libssl ever, and that was it. If there was an update to libz, that didn't require a new download of libssl. After containers, if you build your container for app1 last week, and your container for app2 today, the "FROM ubuntu" likely resolves to a different image. Both your apps now have different "ubuntu" layers, which probably have the same version of libssl, but deduplication of downloads only happens if the whole layer is identical. In essence, we went from downloading 1 copy of libssl (for the host OS only) to 3 copies (host OS + 2 containers w/ different ubuntu bases), and there's no deduplication. That by itself seems like it has to be slower since there's an inherent increase in network bandwidth that has to happen. Even if you have a shared base image, you're at least doubling the downloads of libssl since before you could use the host's copy only. All the items you listed under k8s are things I had before it, excluding "Abstraction of infrastructure". Frankly, if you have a well-made load balancer, it's hard not to have zero-downtime deployments and auto-scaling.
- mixermachine 5y ago
  > We'd just snapshot AMIs to keep OS dependencies fixed This is a good solution, but I would not call it easier. Using docker container feels like installing an app on my smartphone. I choose the version and it will always work like I build it at date x without an additional system. Works for every programming language with every dependency out of the box. Python, Java, Javascript, GO, Ocaml, C, ... > How containers are deployed varies wildly between prod and the local machine I just brought a product of my company to Kubernetes. Run helm upgrade --install . -f dev-values.yaml for dev Run helm upgrade --install . -f prod-values.yaml for prod (of course you need the secrets there. Jenkins has them). My laptop does run an environment with all components of the prod env. Something like email and sap services are of course mocked, but everything else? All on my machine. Why not? I can spin up a new test environment for customers with new settings on the same day. > Both your apps now have different "ubuntu" layers We use a base image that does change not that often. Even if: no problem, the registry is connected via 1000 MBit/s and zero-downtime deployment does its magic so I don't even notice if it takes one or two minutes. Another thing: my node (or VM) libs and the libs of my software should not be connected in any way (at least for me). I want to patch my nodes and my software independently. Different software should also not be bound to libs of another software. > All the items you listed under k8s are things I had before it - How do you easily scale up? Including starting new machines and spinning down machines that are no longer needed - How are multiple software parts executed on one host? - How do you do fail over? I know that everything can be done without Kubernetes. With enough time and money one can create large systems that do this. I spun up a new Kubernetes cluster and ported our product (already containerized) on the cluster in about three months. Really: I also love the classic dev ops and have a proxmox server at home, but Kubernetes just solves many problems at once in a short time.
  - TheDong 5y ago
    > How do you easily scale up? Including starting new machines and spinning down machines that are no longer needed AWS autoscaling groups + cloudwatch for adding and removing machines + checking them into load balancers is something that has worked for longer than K8s has been a thing. > How are multiple software parts executed on one host? systemd units, or for more resource hungry things, multiple autoscaling groups. The overhead of running the kubelet on each host + etcd cluster + apiserver means that I still end up with fewer hosts if I just run each component on every single host vs scaling different deployments independently. It is true that kubernetes might be more resource efficient in some combination of nodes and software, but at under 10 servers, I've always found the overhead of the etcd cluster + apiserver + kubelet to dwarf any savings from not just running 10 copies of my software. > How do you do fail over? The AWS-managed load balancer can fail over based on health checks failing, metrics, or I can add/remove servers from it manually. You can also do DNS health checks, or add a layer of haproxy/nginx/whatever if you want. It's not like k8s has some magic ability to fail over under the hood. It's just using k8s service objects (probably LoadBalancer type), which does the same thing.
deleted 5y ago
[deleted]
jstimpfle 5y ago
My gut feel is that Docker is part of a trend of decreasing software quality. When someone writes "fixed dependencies" I read "developers can more easily add more bloat before the cardhouse tumbles". That happens for example when the "fixed dependencies" are upgraded. I am miserable having to touch all this junk. I feel a project is right when I can just git clone it (a few megabytes of data at most) and am left with a self contained repo that was written with minimal dependencies (optimally stored in-tree), and that can be easily built in seconds with a simple shell script on any reasonably modern system. The bare bones way takes a good amount of initial work, but mostly it's a learning experience. Once one understands a few principles of writing portable software, I'm sure it saves a huge amount of time compared to adding all these shells of junk. -- Oh yeah, I have zero experience about integrating with Kubernetes or whatever. I've been a small time user of Jenkins and CircleCI (unvoluntarily), and when I don't have to set it up and it actually works, it's alright and can help where the developer maybe lacks a bit of discipline (build all targets, run all the tests). But, I doubt these technologies are a replacement for an ergonomic build environment (with simple python build script or even a crude Makefile). Is incremental building a thing on any of this CI pipelines? Because one thing I want is building really really fast, and it's already way too much overhead if I have to go through a git commit to check this stuff. Don't even think about requiring a full rebuild or Docker image build just to get some quick feedback on a code change.
- bottled_poe 5y ago
  Docker pushes developers to think about deployment, not just getting it running on their laptop. And responding to your question: yes - the Docker build process is inherently incremental.
- mixermachine 5y ago
  > The bare bones way takes a good amount of initial work, but mostly it's a learning experience. If my prod environment disappears over night I want to be able to restore everything as fast as possible. Otherwise my boss will be very unhappy and I will be promoted to customer :D. > I feel a project is right when I can just git clone it (a few megabytes of data at most) I don't know your experience but especially small projects often have some difficulties installing all dependencies correctly. The fun starts when you don't run a widely supported distro like Ubuntu. If I want to run an old version of the production software I pull the image version X and execute it. I know that everything in the package has the right version and works like it used to. Tests have been executed and the version in the image has proven to meet the standards back then. > But, I doubt these technologies are a replacement for an ergonomic build environment If you build software for larger customer bases you will most likely encounter some kind of clustering. Most large companies have already adapted this way. For personal and small projects it certainly is overkill. > get some quick feedback on a code change ...wait, do you deploy to prod without tests and no direct way to role back the changes?
wowi42 5y ago
Moving from classic servers to containers you get: - Builds with fixed dependencies that never change. Rollback is easy -> what about VMs? - Easy deployment of a prod environment on a local machine -> yep, that's a nice touch, the only valid point for me! - Fast deployment -> lol no, Im faster with VMs. - Easy automation (use version X with config Y) -> valid for VMs and baremetal too With Kubernetes (or other derivates like Openshift) you get: - Auto scaling -> you can get it with VMs too - Fail over -> you can get it with VMs too - Better resource usage if multiple environments are executed -> you can get it with VMs too - Abstraction of infrastructure -> Should I really write it? - Zero downtime deployment (biggest point for my company, we deploy >3 times per week) -> We do on some specific DC (government style) and we release 10-15 times and day with Bare metal servers and ansible There are applications that do not need Kubernetes or even containers, but is this list really nothing oO? -> None of the arguments convinced me I can imagine that if you use Kubernetes just like a classic cluster it could seem like an unnecesarry added complexity but you gain a lot of things. -> yes, extra cost and extra skills needed
- arpa 5y ago
  Confidently wrong

zz865 5y ago

Yeah we have fixed usage so scaling or easy failover is not something we need.

mixermachine 5y ago
Then I can understand you well. Kubernetes then just provides zero-downtime and additional complexity. When you already have something like a deployment window (like 2 am to 3 am) then ZDT also does not matter.

TheDong 5y ago

Each of those benefits are things I had before using containers or kubernetes, and were simpler. > Builds with fixed dependencies that never change. Rollback is easy Any good build system already did this, such as Bazel, or a Gemfile.lock. We'd just snapshot AMIs to keep OS dependencies fixed... which is what Docker images effectively do. If you re-docker-build the same Dockerfile, it's not like you get the same result of "apt-get install libxml" the next time either. > Easy deployment of a prod environment on a local machine How containers are deployed varies wildly between prod and the local machine. All the things that were hard before are still hard. Things like secrets and external dependencies still usually vary. If prod is a kubernetes environment, getting a suitable k8s environment setup locally sucks, especially since it will probably have a different ingress controller, load balancer setup, storage classes available, resource requests, etc. If prod is kubernetes and local is docker-compose, that honestly seems like just as much work to create a second way to run the stack than just using a bash script + "npm start" or "bundle exec rails server" or whatever. Either way, it's not really a prod environment. It's hard to run identical-to-prod environments locally, and those problems are related to secrets and clouds and such, not due to the lack of containers, in my experience. > Fast deployment In my experience, containers haven't sped up deployment. Let's say you use ubuntu for your host and container's OS. Before containers, this meant you had to download one version of libssl ever, and that was it. If there was an update to libz, that didn't require a new download of libssl. After containers, if you build your container for app1 last week, and your container for app2 today, the "FROM ubuntu" likely resolves to a different image. Both your apps now have different "ubuntu" layers, which probably have the same version of libssl, but deduplication of downloads only happens if the whole layer is identical. In essence, we went from downloading 1 copy of libssl (for the host OS only) to 3 copies (host OS + 2 containers w/ different ubuntu bases), and there's no deduplication. That by itself seems like it has to be slower since there's an inherent increase in network bandwidth that has to happen. Even if you have a shared base image, you're at least doubling the downloads of libssl since before you could use the host's copy only. All the items you listed under k8s are things I had before it, excluding "Abstraction of infrastructure". Frankly, if you have a well-made load balancer, it's hard not to have zero-downtime deployments and auto-scaling.

mixermachine 5y ago
> We'd just snapshot AMIs to keep OS dependencies fixed This is a good solution, but I would not call it easier. Using docker container feels like installing an app on my smartphone. I choose the version and it will always work like I build it at date x without an additional system. Works for every programming language with every dependency out of the box. Python, Java, Javascript, GO, Ocaml, C, ... > How containers are deployed varies wildly between prod and the local machine I just brought a product of my company to Kubernetes. Run helm upgrade --install . -f dev-values.yaml for dev Run helm upgrade --install . -f prod-values.yaml for prod (of course you need the secrets there. Jenkins has them). My laptop does run an environment with all components of the prod env. Something like email and sap services are of course mocked, but everything else? All on my machine. Why not? I can spin up a new test environment for customers with new settings on the same day. > Both your apps now have different "ubuntu" layers We use a base image that does change not that often. Even if: no problem, the registry is connected via 1000 MBit/s and zero-downtime deployment does its magic so I don't even notice if it takes one or two minutes. Another thing: my node (or VM) libs and the libs of my software should not be connected in any way (at least for me). I want to patch my nodes and my software independently. Different software should also not be bound to libs of another software. > All the items you listed under k8s are things I had before it - How do you easily scale up? Including starting new machines and spinning down machines that are no longer needed - How are multiple software parts executed on one host? - How do you do fail over? I know that everything can be done without Kubernetes. With enough time and money one can create large systems that do this. I spun up a new Kubernetes cluster and ported our product (already containerized) on the cluster in about three months. Really: I also love the classic dev ops and have a proxmox server at home, but Kubernetes just solves many problems at once in a short time.
- TheDong 5y ago
  > How do you easily scale up? Including starting new machines and spinning down machines that are no longer needed AWS autoscaling groups + cloudwatch for adding and removing machines + checking them into load balancers is something that has worked for longer than K8s has been a thing. > How are multiple software parts executed on one host? systemd units, or for more resource hungry things, multiple autoscaling groups. The overhead of running the kubelet on each host + etcd cluster + apiserver means that I still end up with fewer hosts if I just run each component on every single host vs scaling different deployments independently. It is true that kubernetes might be more resource efficient in some combination of nodes and software, but at under 10 servers, I've always found the overhead of the etcd cluster + apiserver + kubelet to dwarf any savings from not just running 10 copies of my software. > How do you do fail over? The AWS-managed load balancer can fail over based on health checks failing, metrics, or I can add/remove servers from it manually. You can also do DNS health checks, or add a layer of haproxy/nginx/whatever if you want. It's not like k8s has some magic ability to fail over under the hood. It's just using k8s service objects (probably LoadBalancer type), which does the same thing.

deleted 5y ago

[deleted]

jstimpfle 5y ago

My gut feel is that Docker is part of a trend of decreasing software quality. When someone writes "fixed dependencies" I read "developers can more easily add more bloat before the cardhouse tumbles". That happens for example when the "fixed dependencies" are upgraded. I am miserable having to touch all this junk. I feel a project is right when I can just git clone it (a few megabytes of data at most) and am left with a self contained repo that was written with minimal dependencies (optimally stored in-tree), and that can be easily built in seconds with a simple shell script on any reasonably modern system. The bare bones way takes a good amount of initial work, but mostly it's a learning experience. Once one understands a few principles of writing portable software, I'm sure it saves a huge amount of time compared to adding all these shells of junk. -- Oh yeah, I have zero experience about integrating with Kubernetes or whatever. I've been a small time user of Jenkins and CircleCI (unvoluntarily), and when I don't have to set it up and it actually works, it's alright and can help where the developer maybe lacks a bit of discipline (build all targets, run all the tests). But, I doubt these technologies are a replacement for an ergonomic build environment (with simple python build script or even a crude Makefile). Is incremental building a thing on any of this CI pipelines? Because one thing I want is building really really fast, and it's already way too much overhead if I have to go through a git commit to check this stuff. Don't even think about requiring a full rebuild or Docker image build just to get some quick feedback on a code change.

bottled_poe 5y ago
Docker pushes developers to think about deployment, not just getting it running on their laptop. And responding to your question: yes - the Docker build process is inherently incremental.
mixermachine 5y ago
> The bare bones way takes a good amount of initial work, but mostly it's a learning experience. If my prod environment disappears over night I want to be able to restore everything as fast as possible. Otherwise my boss will be very unhappy and I will be promoted to customer :D. > I feel a project is right when I can just git clone it (a few megabytes of data at most) I don't know your experience but especially small projects often have some difficulties installing all dependencies correctly. The fun starts when you don't run a widely supported distro like Ubuntu. If I want to run an old version of the production software I pull the image version X and execute it. I know that everything in the package has the right version and works like it used to. Tests have been executed and the version in the image has proven to meet the standards back then. > But, I doubt these technologies are a replacement for an ergonomic build environment If you build software for larger customer bases you will most likely encounter some kind of clustering. Most large companies have already adapted this way. For personal and small projects it certainly is overkill. > get some quick feedback on a code change ...wait, do you deploy to prod without tests and no direct way to role back the changes?

wowi42 5y ago

Moving from classic servers to containers you get: - Builds with fixed dependencies that never change. Rollback is easy -> what about VMs? - Easy deployment of a prod environment on a local machine -> yep, that's a nice touch, the only valid point for me! - Fast deployment -> lol no, Im faster with VMs. - Easy automation (use version X with config Y) -> valid for VMs and baremetal too With Kubernetes (or other derivates like Openshift) you get: - Auto scaling -> you can get it with VMs too - Fail over -> you can get it with VMs too - Better resource usage if multiple environments are executed -> you can get it with VMs too - Abstraction of infrastructure -> Should I really write it? - Zero downtime deployment (biggest point for my company, we deploy >3 times per week) -> We do on some specific DC (government style) and we release 10-15 times and day with Bare metal servers and ansible There are applications that do not need Kubernetes or even containers, but is this list really nothing oO? -> None of the arguments convinced me I can imagine that if you use Kubernetes just like a classic cluster it could seem like an unnecesarry added complexity but you gain a lot of things. -> yes, extra cost and extra skills needed

arpa 5y ago
Confidently wrong