8 ms·

In my opinion, the most common mistake is not in the article : using kubernetes when you don't need to. Kubernetes has a lot of pros or the papers but in pract

In my opinion, the most common mistake is not in the article : using kubernetes when you don't need to. Kubernetes has a lot of pros or the papers but in practice it's not worth it for most small and medium companies.

balfirevic 7y ago
Do you also include managed kubernetes offerings, such as from Digital Ocean, in that assessment?
- blinkingled 7y ago
  Not the OP but yes if cost is a factor. As far as I know no managed K8S offerings are cheap.
  - deleted 7y ago
    [deleted]
  - watermelon0 7y ago
    - DigitalOcean offers free clusters. - Azure (still?) offers free clusters. - GCP offers one free single-zone cluster. If you need more than DO can offer in terms of compute instances, you can probably afford GKE/EKS, which is around 75$/month. --- Running highly-available control plane (K8s masters & etcd) by yourself is NOT cheaper than using EKS. To achive high availability, EKS runs 3 masters and 3 etcd instances, in different availability zones. Provisioning 3 t3.medium instances (4 GB of memory and 2 CPUs) would cost the same as a completely managed EKS. Not to mention the manual work you need to setup, maintain and upgrade such instances.
- speedgoose 7y ago
  Oh yes. Managed kubernetes is full of various issues. Some major cloud providers sell very poor managed kubernetes. If someone knows about a reliable managed kubernetes, please let me know.
  - politelemon 7y ago
    Can you elaborate on the cloud providers selling poorly managed k8s - are they all problematic? I have no experience with cloud provided k8s.
- jcrawfordor 7y ago
  Honestly my experience has been that managed k8s is often more complicated from a developer perspective than just k8s - sure, you don't have to deal with setting it up, but you have to figure out how all the 'management' and provider-specific features work, and they often seem pretty clumsily integrated.
  - geerlingguy 7y ago
    And in many cases features that would help your use case aren't enabled on that platform, or are in a release that's still a year or two from being supported on that platform... I'm looking at you, EKS.
  - GordonS 6y ago
    I really wish cloud providers would offer a managed Docker Swarm service - it's so much simpler than k8s, but not having to manage the underlying machines would be great. I might be remembering wrongly, but I think Azure actually used to offer multiple managed orchestration platforms, including DCOS and Swarm, before they went all-in on k8s.
nyi 7y ago
Are there good resources for making that decision according to good criteria?
- therein 7y ago
  A simple rule of thumb is the number of services you have. For instance, my current employer has been working on getting everything on Kubernetes for the last year or so and we have two services... the frontend server-side renderer, and API.
  - coredog64 6y ago
    /snark So is that a cluster per app or a namespace per app?
- Niksko 6y ago
  The criteria is basically: do you have the time, people, expertise and willingness to build a custom dev platform on top of Kubernetes? If yes, Kubernetes might make sense for you, if your company is large enough that a PaaS like Heroku is prohibitively expensive. If no, you don't want Kubernetes. https://twitter.com/kelseyhightower/status/935252923721793536?s=19 https://twitter.com/kelseyhightower/status/93525292372179353...
rwoll 7y ago
This was my thought exactly. The article is great assuming you need to use k8s, but does leave out the important question: does your project or product require k8s and all the overhead it unavoidably entails? Amazon's Elastic Container Service (ECS) on Fargate deployment type is probably a better option much of the time. Until you maintain your own k8s cluster (including the hosted variants on AWS, GCP, etc.) you might not realize how complex configuring k8s is. While the AWS's ECS service may be more limited, I've found it leaves less room to do the wrong thing. Unfortunately, the documentation on ECS and the community support is inferior to k8s, but I'll accept that if I don't have to spend a whole day researching what service mesh to use with k8s and how to configure a load balancer and SSL certs.
- marekaf 6y ago
  You are right, but talking about whether to use k8s or not would make the article 2-3x longer. It is a totally different topic. ECS (with Fargate or EC2) is awesome but one might argue that it is still an overkill for a lot of people. Some people look for scaling, immutability, effective resource utilization and they think they NEED containers. That is not entirely true. A lot of people would do totally ok with one EC2 instance. Either way, kubernetes is awesome, there is a big community behind it maintaining a lot of tooling, docs, how-to guides, etc. That is very valuable for a lot of people too :)
  - rwoll 6y ago
    > You are right, but talking about whether to use k8s or not would make the article 2-3x longer. It is a totally different topic. Sorry—my comment was written poorly. I think the article is good as is and agree k8s or not is a different topic. My comment intended to be a reminder for all of us reading it that we should ask more questions before choosing a technology. I’ve found a trend in engineering (including with myself), we sometimes forget to take a step back and see the forest.
- coredog64 6y ago
  Unless something has changed recently, Fargate is a terrible choice for something that you want up 24x7. IIRC it’s like 2x the cost of similar on demand EC2 instances. It’s better for discrete tasks that can’t run in Lambda because timeout or possibly peak load (although EC2 is probably just as easy if not easier to scale). But I generally agree with the premise that ECS is a better starting point for people just dipping their toe into containers.
deleted 7y ago
[deleted]
bufferoverflow 7y ago
Most of the time you don't even need a cluster of any kind. We live in a world where you can spin up a server with 3TB of RAM and 128 cores whenever you want, and it will cost you less than a senior developer.
- user5994461 6y ago
  https://aws.amazon.com/ec2/pricing/on-demand/ https://aws.amazon.com/ec2/pricing/on-demand/ x1e.16xlarge, 64 CPU, 2TB of RAM, $10k per month.
  - dijit 6y ago
    So if you pay a senior developer less than 120k then the parent is right.
    - bufferoverflow 6y ago
      It costs the employer a lot more than just the salary to have a salaried developer.
- aphistic 6y ago
  What do you do when that one machine goes down?

balfirevic 7y ago

Do you also include managed kubernetes offerings, such as from Digital Ocean, in that assessment?

blinkingled 7y ago
Not the OP but yes if cost is a factor. As far as I know no managed K8S offerings are cheap.
- deleted 7y ago
  [deleted]
- watermelon0 7y ago
  - DigitalOcean offers free clusters. - Azure (still?) offers free clusters. - GCP offers one free single-zone cluster. If you need more than DO can offer in terms of compute instances, you can probably afford GKE/EKS, which is around 75$/month. --- Running highly-available control plane (K8s masters & etcd) by yourself is NOT cheaper than using EKS. To achive high availability, EKS runs 3 masters and 3 etcd instances, in different availability zones. Provisioning 3 t3.medium instances (4 GB of memory and 2 CPUs) would cost the same as a completely managed EKS. Not to mention the manual work you need to setup, maintain and upgrade such instances.
speedgoose 7y ago
Oh yes. Managed kubernetes is full of various issues. Some major cloud providers sell very poor managed kubernetes. If someone knows about a reliable managed kubernetes, please let me know.
- politelemon 7y ago
  Can you elaborate on the cloud providers selling poorly managed k8s - are they all problematic? I have no experience with cloud provided k8s.
jcrawfordor 7y ago
Honestly my experience has been that managed k8s is often more complicated from a developer perspective than just k8s - sure, you don't have to deal with setting it up, but you have to figure out how all the 'management' and provider-specific features work, and they often seem pretty clumsily integrated.
- geerlingguy 7y ago
  And in many cases features that would help your use case aren't enabled on that platform, or are in a release that's still a year or two from being supported on that platform... I'm looking at you, EKS.
- GordonS 6y ago
  I really wish cloud providers would offer a managed Docker Swarm service - it's so much simpler than k8s, but not having to manage the underlying machines would be great. I might be remembering wrongly, but I think Azure actually used to offer multiple managed orchestration platforms, including DCOS and Swarm, before they went all-in on k8s.

nyi 7y ago

Are there good resources for making that decision according to good criteria?

therein 7y ago
A simple rule of thumb is the number of services you have. For instance, my current employer has been working on getting everything on Kubernetes for the last year or so and we have two services... the frontend server-side renderer, and API.
- coredog64 6y ago
  /snark So is that a cluster per app or a namespace per app?
Niksko 6y ago
The criteria is basically: do you have the time, people, expertise and willingness to build a custom dev platform on top of Kubernetes? If yes, Kubernetes might make sense for you, if your company is large enough that a PaaS like Heroku is prohibitively expensive. If no, you don't want Kubernetes. https://twitter.com/kelseyhightower/status/935252923721793536?s=19 https://twitter.com/kelseyhightower/status/93525292372179353...

rwoll 7y ago

This was my thought exactly. The article is great assuming you need to use k8s, but does leave out the important question: does your project or product require k8s and all the overhead it unavoidably entails? Amazon's Elastic Container Service (ECS) on Fargate deployment type is probably a better option much of the time. Until you maintain your own k8s cluster (including the hosted variants on AWS, GCP, etc.) you might not realize how complex configuring k8s is. While the AWS's ECS service may be more limited, I've found it leaves less room to do the wrong thing. Unfortunately, the documentation on ECS and the community support is inferior to k8s, but I'll accept that if I don't have to spend a whole day researching what service mesh to use with k8s and how to configure a load balancer and SSL certs.

marekaf 6y ago
You are right, but talking about whether to use k8s or not would make the article 2-3x longer. It is a totally different topic. ECS (with Fargate or EC2) is awesome but one might argue that it is still an overkill for a lot of people. Some people look for scaling, immutability, effective resource utilization and they think they NEED containers. That is not entirely true. A lot of people would do totally ok with one EC2 instance. Either way, kubernetes is awesome, there is a big community behind it maintaining a lot of tooling, docs, how-to guides, etc. That is very valuable for a lot of people too :)
- rwoll 6y ago
  > You are right, but talking about whether to use k8s or not would make the article 2-3x longer. It is a totally different topic. Sorry—my comment was written poorly. I think the article is good as is and agree k8s or not is a different topic. My comment intended to be a reminder for all of us reading it that we should ask more questions before choosing a technology. I’ve found a trend in engineering (including with myself), we sometimes forget to take a step back and see the forest.
coredog64 6y ago
Unless something has changed recently, Fargate is a terrible choice for something that you want up 24x7. IIRC it’s like 2x the cost of similar on demand EC2 instances. It’s better for discrete tasks that can’t run in Lambda because timeout or possibly peak load (although EC2 is probably just as easy if not easier to scale). But I generally agree with the premise that ECS is a better starting point for people just dipping their toe into containers.

deleted 7y ago

[deleted]

bufferoverflow 7y ago

Most of the time you don't even need a cluster of any kind. We live in a world where you can spin up a server with 3TB of RAM and 128 cores whenever you want, and it will cost you less than a senior developer.

user5994461 6y ago
https://aws.amazon.com/ec2/pricing/on-demand/ https://aws.amazon.com/ec2/pricing/on-demand/ x1e.16xlarge, 64 CPU, 2TB of RAM, $10k per month.
- dijit 6y ago
  So if you pay a senior developer less than 120k then the parent is right.
  - bufferoverflow 6y ago
    It costs the employer a lot more than just the salary to have a salaried developer.
aphistic 6y ago
What do you do when that one machine goes down?