5 ms·

You can't make the complexity disappear. Kubernetes just offers a very standardized, stable and relatively polished way to handle it. The other alternatives, l

You can't make the complexity disappear. Kubernetes just offers a very standardized, stable and relatively polished way to handle it. The other alternatives, like running your own orchestration system, are just as complex if not more so, although you might be more familiar with it since you built it. That being said, many companies don't need any of it in the first place for their scale, and that's probably the biggest issue with K8S today.

yjftsjthsd-h 6y ago
I'd say it the other way; k8s has to do all the things that another management system would have to do, but sufficiently tied together that you can't ease into it as needed.
- manigandham 6y ago
  I guess that depends primarily on whether you're installing and operating K8S yourself. Use something like GKE and it's a very seamless experienced, with AKS getting pretty good and the rest being rather crummy. Once you have a managed cluster, the deploying apps is fairly easy. A single container/pod is a 1-liner and you can work your way up from there.
  - yjftsjthsd-h 6y ago
    That's fair; all of this is heavily influenced by your operating environment. If you can run GKE or if you have an ops team to deal with that stuff, then yeah k8s is great. Unfortunately, I'm part of the ops team, and our company is too small to have a dedicated k8s team and too low-budget to (likely) do well with a managed service (we get absurd value per money out of bare metal servers, which is very much a tradeoff that is sometimes painful). So to me, k8s looks like a very iffy tradeoff. Bigger company, bigger budget, different constraints? Yeah, k8s would be great.
    - mwcampbell 6y ago
      > we get absurd value per money out of bare metal servers, which is very much a tradeoff that is sometimes painful What do you find most painful about your use of bare-metal servers? The thing that I like most about a hyperscale cloud provider is the level of redundancy, including even multiple data centers per region, and their built-in health checks and recovery (e.g. through auto-scaling groups) based on that redundancy. With bare-metal servers, I'd have to cobble together my own failover system for the occasional time when one of those servers goes down or becomes unreachable due to a network issue. And of course, I'd probably find out that my home-made failover system doesn't actually work at the worst possible time.
      - yjftsjthsd-h 6y ago
        I don't know that it's a single big thing; it is indeed many small things that we have to manage ourselves. We backup the databases with our own scripts, we failover manually (I miss RDS), we use an overlay network because no VPC, and deployments involve ansible running docker-compose. There's basically no elasticity; we provision massive bare metal servers with fixed memory and disk installed. But, it is dirt cheap, so we manage, and all the pieces are small and easy enough to use.
daze42 6y ago
It's impossible to make essential complexity disappear but it is certainly possible to reduce incidental complexity. Most software is much more complex than it needs to be and Kubernetes is no exception.
- CraigJPerry 6y ago
  Before k8s every serious shop automated the crap out of their infra. Jump/kickstart recipes, rolling cluster patching that split RAID mirrors before applying, blue/green deployment scripts to tickle the loadbalancer, cron jobs to purge old releases ... That stack is super complex and utterly bespoke to the company. With k8s it’s standardised and usually better quality.
  - folmar 6y ago
    It's on a path to standardised, but not there yet: etcd vs. others, different ingress controllers, providers replacing most of network parts, storage is bumpy/not so standard, deploy may be kubectl apply/helm/operator. I would really appreciate a more mature ecosystem.
rodgerd 6y ago
> That being said, many companies don't need any of it in the first place for their scale, and that's probably the biggest issue with K8S today. Yes. I helped push out a k8s platform where I work, and it's been running well in production for ~3 years. We most definitely did not start with "we want to deploy k8s". We started with "we're being asked to meet certain business requirements", which lead to "we will need to change how we do some of our development and deployment", which lead to "our platform will require these characteristics." The easiest way to get those turned out to be buying a packaged k8s from a vendor (OpenShift, not that it matters; there's plenty of options). Most of the pain with k8s seems to come down to people wanting to polish their CVs (by "having done k8s"), or people who sneer at packaged/hosted solutions because they want to build a cottage industry of building k8s services from scratch because it's their idea of a good time. Unsurprisingly both take orgs down the route of pain, money, and regrets. Oftentimes the people driving this decision then prefer to say the problems is k8s, or containers. Or microservices or whatever fad it was they were chasing without understanding whether it would be a good idea.
zapf 6y ago
k8s and even docker are trying to solve a problem not many people will ever face. However, being the sexy new things loads of people get sucked into integrating it into their stack from the word go. Being reluctant to adopt new technology unless you really really need might be a more sensible thing to do.

yjftsjthsd-h 6y ago

I'd say it the other way; k8s has to do all the things that another management system would have to do, but sufficiently tied together that you can't ease into it as needed.

manigandham 6y ago
I guess that depends primarily on whether you're installing and operating K8S yourself. Use something like GKE and it's a very seamless experienced, with AKS getting pretty good and the rest being rather crummy. Once you have a managed cluster, the deploying apps is fairly easy. A single container/pod is a 1-liner and you can work your way up from there.
- yjftsjthsd-h 6y ago
  That's fair; all of this is heavily influenced by your operating environment. If you can run GKE or if you have an ops team to deal with that stuff, then yeah k8s is great. Unfortunately, I'm part of the ops team, and our company is too small to have a dedicated k8s team and too low-budget to (likely) do well with a managed service (we get absurd value per money out of bare metal servers, which is very much a tradeoff that is sometimes painful). So to me, k8s looks like a very iffy tradeoff. Bigger company, bigger budget, different constraints? Yeah, k8s would be great.
  - mwcampbell 6y ago
    > we get absurd value per money out of bare metal servers, which is very much a tradeoff that is sometimes painful What do you find most painful about your use of bare-metal servers? The thing that I like most about a hyperscale cloud provider is the level of redundancy, including even multiple data centers per region, and their built-in health checks and recovery (e.g. through auto-scaling groups) based on that redundancy. With bare-metal servers, I'd have to cobble together my own failover system for the occasional time when one of those servers goes down or becomes unreachable due to a network issue. And of course, I'd probably find out that my home-made failover system doesn't actually work at the worst possible time.
    - yjftsjthsd-h 6y ago
      I don't know that it's a single big thing; it is indeed many small things that we have to manage ourselves. We backup the databases with our own scripts, we failover manually (I miss RDS), we use an overlay network because no VPC, and deployments involve ansible running docker-compose. There's basically no elasticity; we provision massive bare metal servers with fixed memory and disk installed. But, it is dirt cheap, so we manage, and all the pieces are small and easy enough to use.

daze42 6y ago

It's impossible to make essential complexity disappear but it is certainly possible to reduce incidental complexity. Most software is much more complex than it needs to be and Kubernetes is no exception.

CraigJPerry 6y ago
Before k8s every serious shop automated the crap out of their infra. Jump/kickstart recipes, rolling cluster patching that split RAID mirrors before applying, blue/green deployment scripts to tickle the loadbalancer, cron jobs to purge old releases ... That stack is super complex and utterly bespoke to the company. With k8s it’s standardised and usually better quality.
- folmar 6y ago
  It's on a path to standardised, but not there yet: etcd vs. others, different ingress controllers, providers replacing most of network parts, storage is bumpy/not so standard, deploy may be kubectl apply/helm/operator. I would really appreciate a more mature ecosystem.

rodgerd 6y ago

> That being said, many companies don't need any of it in the first place for their scale, and that's probably the biggest issue with K8S today. Yes. I helped push out a k8s platform where I work, and it's been running well in production for ~3 years. We most definitely did not start with "we want to deploy k8s". We started with "we're being asked to meet certain business requirements", which lead to "we will need to change how we do some of our development and deployment", which lead to "our platform will require these characteristics." The easiest way to get those turned out to be buying a packaged k8s from a vendor (OpenShift, not that it matters; there's plenty of options). Most of the pain with k8s seems to come down to people wanting to polish their CVs (by "having done k8s"), or people who sneer at packaged/hosted solutions because they want to build a cottage industry of building k8s services from scratch because it's their idea of a good time. Unsurprisingly both take orgs down the route of pain, money, and regrets. Oftentimes the people driving this decision then prefer to say the problems is k8s, or containers. Or microservices or whatever fad it was they were chasing without understanding whether it would be a good idea.

zapf 6y ago

k8s and even docker are trying to solve a problem not many people will ever face. However, being the sexy new things loads of people get sucked into integrating it into their stack from the word go. Being reluctant to adopt new technology unless you really really need might be a more sensible thing to do.