17 ms·

Do I Need Kubernetes?

206 points by mocko 6y ago

elliebike 6y ago
Honestly nobody needs k8s, but nobody really needs anything. If you know it and you know it suits your needs, then sure! Go for it! Have fun. If you don't know it, then learn it first. Then you can properly evaluate whether it suits your requirements or not. There's huge value in choosing something that your team already knows + is familiar with I'd totally advocate for learning something new, but solving a business problem probably isn't the best time for this :) I think the article gave a great overview though, and is perhaps a good way to decide what to learn next
mrweasel 6y ago
I only have one tiny detail: Bursty traffic means that your cluster needs to be able to deal with the peaks. If you running an on prem Kubernetes cluster, then there is no savings, unless you can use the capacity for something else during non peak periods. The scaling, and potential savings is a cloud feature, not a feature of Kubernetes.
- tstrimple 6y ago
  It also enables you to achieve app density. Many of the companies I've been working with lately have large batch processes on nightly / weekly / monthly basis. For some reason, each job previously was setup on on-prem hardware dedicated per workload. Using Kubernetes and scheduling jobs to manage capacity has enabled us to reduce the number of on-prem servers substantially as part of the migration plan for the cloud.
  - halbritt 6y ago
    Precisely. In my experience hosting services in VMs on-prem I was able to achieve roughly 30% efficiency across 3k instances and hundreds of nodes. In my experience hosting microservices on k8s, I was able to achieve 80% efficiency across hundreds of nodes. Both were the result of a great deal of work to optimize efficiency. In this case, I use the word "efficiency" to refer to a blend of CPU and memory utilization.
- cat199 6y ago
  > no savings This only relates to hardware cost - tooling impact can provide cost improvements if it enhances productivity (not always clear for k8s case)
- aliswe 6y ago
  I would nitpick that and say "if you are running a cluster with a static number of worker nodes". If I would have a cloud hosted k8s cluster and don't configure any auto scaling, the situation would be equivalent to me buying x number of physical machines and hosting them as worker nodes on prem. No ability to scale to "bursty traffic".
- ec109685 6y ago
  If the burst is for all apps at once, then yes. But if you need to handle temporary bursts of a single app among manh, then hosting them in a heterogenous cluster will give you benefit.
deleted 6y ago
[deleted]
ramraj07 6y ago
Not a single mention of elastic beanstalk or App Engine? The best middle ground for small teams who just want one reliable website with minimal scaling (and who can't just choose a nom-aws service).
- rudolph9 6y ago
  Do you have any examples of non-proprietary solutions?
  - harpratap 6y ago
    Knative + buildpacks is one. CloudFoundry is open source too (although CF now runs on Kubernetes I think)
    - ramanujank 6y ago
      Cloud foundry offers their old(er) architecture as well as 2 ways to get the “cf push” experience over Kubernetes. Check out KubeCF and cf-for-k8s. Both of these are open source projects that you can deploy to a Kubernetes infrastructure of choice. [1] https://github.com/cloudfoundry-incubator/kubecf https://github.com/cloudfoundry-incubator/kubecf [2] https://github.com/cloudfoundry/cf-for-k8s https://github.com/cloudfoundry/cf-for-k8s
- bpicolo 6y ago
  I've found deployments to be much more slow than I'd hope for in both
- cloudking 6y ago
  We ran on AppEngine from launch to exit, great experience. Never had to worry about configurations, environments, maintenance, reliability, scalability etc. We just focused on building our product.
- te_chris 6y ago
  We tried to use app engine and it was a nightmare for us. K8s has a ramp, but it’s been incredibly smooth now that we’re setup and going.
gravypod 6y ago
> Run your applications in Docker from day 1 (with docker-compose it’s as valuable for dev as it is for production) and think carefully before letting your applications store local state. I think this is the key take away for many startups. Get it so you: 1. Have a single-command way to bring up your entire backend 2. That command can run on your dev machine 3. You document all of the settings for your containers and deps Once you have that in a docker-compose.yml file migrating to something like kube when you need health checks, autoscaling, etc is easy. The only thing you must be critical of is bashing people over the head to make everything you run in your company run in this environment you've created. No special deployments. Establish 1 method with 1 tool and go from there. Every company I've worked at I've brought a single command `docker-compose up -d` workflow and people were amazed by the productivity gains. No more remembering ports (container_name.local.company.com), no more chasing down configuration files and examples, no more talking to shared DBs, etc.
- thom 6y ago
  Can I ask people who _don't_ use Kubernetes and maybe have architectures built on proprietary cloud services: how do you manage this?
  - gravypod 6y ago
    Depends on what cloud services you use. I am very opposed to using any cloud service that I can't self host in some way or another. That's a sure fire way to end up screwed. The main cloud services I use at work are: 1. S3/Azure Blob Store 2. DocumentDB/CosmoDB/Cloud Atlas 3. Aurora/A for PG DB 4. ElastiCache/some complex name in azure All of these data stores can be hosted locally using minio for s3, MongoDB for the mongo implementations, and postgres/mysql for the RDBMSs, redis/memcached for ElastiCache. If there was ever a case that I needed to ever use a real - full vendor lockin - solution I would hide it behind an API of my own and configure that API to store data locally. For example, the Azure Blob Store local container is completely broken for any sensible usage and I've been on an github issue thread for ~3 months waiting for them to resolve the issue (you're forced to link your netns to that container to talk to it as the application clients only works over 127.0.0.1). So essentially I have the following settings: - SERVICE_STORAGE_DRIVER=azure/fs - SERVICE_FS_LOCATION=/data - SERVICE_AZURE_CONTAINER=.... - SERVICE_AZURE_....=.... So in development it runs in fs mode, in prod it runs in azure mode, you mock the azure blob client in unit test to make sure that code works.
irontinkerer 6y ago
> To make a cluster useful for the average workload a menagerie of add-ons will be required. Some of them almost everyone uses, others are somewhat niche. This is the concern I have with k8s. All this complexity introduces operational and security concerns, while adding more work to do before you can just deploy business value (compared to launching on standard auto-scaling cloud instances)
- orf 6y ago
  Like most infrastructure work there is an upfront cost, but after paying that the result enables you to ship value faster and easier.
- gravypod 6y ago
  If you are using a managed kubernetes cluster from a cloud provider you mostly don't need to worry about these sorts of things. If you're not, and deploying to bare metal, the main things you need to worry about are: load balancers, storage & monitoring. If you're large enough that you can effectively run kube on bare metal you probably have enterprise solutions for load balacing [0], storage [1] & monitoring your applications that you've already validated as being secure/stable. If you want to go all out you can also grab an operator to manage rolling out databases for you (postgres [2], mongo, etc). A lot of the complexity people bump into with kube is really poorly planned out tools like Istio that have way too many features, a very overly complex mode of operation (out of the box it breaks CronJobs!!!), and very sub-standard community documentation. If you avoid Istio, or anything that injects sidecars and initcontainers, you'll find the experience enjoyable. [0] - https://clouddocs.f5.com/containers/v2/kubernetes/ https://clouddocs.f5.com/containers/v2/kubernetes/ [1] - https://www.netapp.com/us/kubernetes-storage.aspx https://www.netapp.com/us/kubernetes-storage.aspx [2] - https://github.com/CrunchyData/postgres-operator https://github.com/CrunchyData/postgres-operator
- solatic 6y ago
  As a rule of thumb, the quality of engineering of the core Kubernetes distribution is rock solid and incredible, but anything that's not is in varying stages of maturity. How much Kubernetes-adjacent code you actually need to adopt (and therefore, how much risk you take on) depends from project to project and organization to organization.
whotheffknows 6y ago
No.
peterwwillis 6y ago
Here's a quick reference: 1) Are you on AWS? Then you don't need Kubernetes. Use Fargate. 2) Are you on Google Cloud? Then you don't need Kubernetes. Use Cloud Run. 3) Are you on Azure? Then you don't need Kubernetes. Use Azure Container Instances. 4) Are you on a PaaS like Heroku? Then you don't need Kubernetes. 5) Are you on a random VPC provider / bare metal machines? You could probably still do without Kubernetes using Docker Swarm (apparently it's not dead!), Nomad, Mesos DC/OS, or a standard Linux box and systemd (or some other process or cluster scheduler). 6) Do you need to solve the bin-packing problem? Do you need to self-host a service mesh of microservices in multiple colocated regions? Do you need a fully automated redundant fault-tolerant network of disposable nodes to constantly reschedule different versions of applications with stringent RBACs, scheduled tasks, dynamic resource allocation, and do you have about a million dollars to spend on building and maintaining it all? Then you need Kubernetes.
- jrott 6y ago
  I get that feeling however I know for fargate there are some vertical scaling limitations that don't exist with kubernetes. Also at this point there is a ton of stuff out there that assumes you are deploying to kubernetes and makes that the path of least resistance.
  - otterley 6y ago
    > I know for fargate there are some vertical scaling limitations that don't exist with kubernetes To overcome those limitations, you can use EC2 Container Instances with ECS. And instance auto scaling can now be automated for you via ECS Capacity Providers.
- lazyant 6y ago
  You may want to qualify these absolutes, as in "you may not need k8s, consider blah". For ex I run on k8s and can't use Fargate, same for GCP and Azure.
- halbritt 6y ago
  > do you have about a million dollars to spend on building and maintaining it all? Then you need Kubernetes. I think you're overstating the investment necessary to overcome the initial complication of Kubernetes and also understating the benefit of being on a platform with a massive and thriving community behind it. As an example, in a prior role, there were a set of data engineers that would receive data in the form of MS SQL server backups from which these engineers would need to query and transform data on an exploratory rather than production basis. Certainly one could use an "undifferentiated" service from a cloud provider, but it was also a roughly 5-minute process for me to use the rather high quality helm chart and docker image commonly available to stand up a new service for the benefit of each engineer that had the need. The process of creating that automation necessary to deploy the helm chart and restore the backups took approximately one hour and could be repeated ad nauseum in the aforementioned 5-minute time period. There are many, many other examples of this. Want a data-science platform complete out of the box with no vendor lock-in, how about data8.org? The list goes on.
dennis_jeeves 6y ago
No
- kseistrup 6y ago
  Exactly! Betteridge’s law of headlines: Any headline that ends in a question mark can be answered by the word no. ⌘ https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
  - 2mol 6y ago
    I get it, but it's not really doing the author justice, who's trying to give a quite nuanced overview of where kubernetes is a good fit, and where it isn't. As somebody who's pretty skeptical about onboarding the big lump of complexity that is k8s, I really appreciated the information in the article.
    - dennis_jeeves 6y ago
      You see, any pile of junk(Kubernetes being one of them) can have a use case if you searched hard enough. Do not learn/work with Kubernetes unless your livelihood depends on it.
  - dang 6y ago
    Mr. Betteridge retired after a thousand of these or so. https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=betteridge&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu.... Interestingly, it was pg who got it going on HN: https://news.ycombinator.com/item?id=4092880 https://news.ycombinator.com/item?id=4092880 I once used the joke about Mr. Betteridge retiring and got a reply pointing out that he hasn't: https://news.ycombinator.com/item?id=10393754 https://news.ycombinator.com/item?id=10393754.
kanobo 6y ago
When a newspaper headline or blog post title ends in a question mark, the answer is almost always 'no'.
- dang 6y ago
  https://news.ycombinator.com/item?id=24179969 https://news.ycombinator.com/item?id=24179969
spost 6y ago
I work for a startup whose product is small (half a dozen servers, if relatively beefy ones) clusters that will be run on-prem by customers, at least sometimes in a low-to-no-touch capacity. Most of our application components are micro-ish services that are run on all hosts in the cluster for either extra capacity or fault tolerance. We currently run everything on mesos/marathon, but are looking to switch away from it. K8s is kinda the “default” option, and is potentially appealing to some potential acquirers and investors. But I never really see k8s being talked about in that context of “physical hardware that’s on prem, but not on MY prem.” Is there a reason for that? If we go with k8s is it going to bite us? Does anyone have experience with something like that they could share?
- dilyevsky 6y ago
  You maybe interested in metal3.io And yes it will “bite” you but so will everything else.
- lazyant 6y ago
  I have some experience with this. Way I see it, if you have your software on somebody else's on-prem and move to k8s, more importantly than replacing the stack between Linux and your app is a change in mentality and demarcation of responsibilities, as in now your app "runs on k8s" and your clients are responsible for that layer (or they can contract that out) and you are responsible only for your app. It helps abstract out everything in the stack below your app and easier conceptually on everybody; now the clients can train in k8s and use same set of tooling like prometheus/grafana etc that usually go with k8s, same or similar RBAC access etc. OTOH realistic expectations need to be set; not because it's k8s there's going to be no problems or the learning/adaptation won't come without some pain. I suggest writing down some standard procedures for your clients like upgrades etc, pick same set of tooling for all of them (like same dashboard, same logging/monitoring/alerting etc) as a way to homeganize ("standardize") all of them. Feel free to email me.
- solatic 6y ago
  > But I never really see k8s being talked about in that context of “physical hardware that’s on prem, but not on MY prem.” Is there a reason for that? If we go with k8s is it going to bite us? Does anyone have experience with something like that they could share? Kubernetes provides a leaky abstraction above the underlying hardware - the storage and networking are going to be different depending on who is maintaining the Kubernetes cluster. Kubernetes's strength is that it acknowledges the leakiness of the abstraction and makes it explicit. If your customer uses a specific networking and storage provider, Kubernetes makes it easier for you to say (or not) that you have certified your product for those networking and storage providers, and here's what the manifests look like, because there's a standard way of configuring the application to work with that networking (CNI, which powers the standard Service as well as maybe NetworkPolicies) and storage (CSI, specifically StorageClass) provider. If you just provide Docker images, or VM appliances, then Murphy promises you that you're going to get frustrated support calls from customers saying "your application is slow and we don't understand why." Good luck then.
vasilakisfil 6y ago
For small apps and projects that continuous delivery is not required I would start with lxd. I think it's the easiest way to containerize an app. Once that feels too little, I would start looking at docker, and only when docker feels again too little, to kubernetes. So in essence, for 95% of the apps/people, the answer is no.
- marcc 6y ago
  Can you elaborate on why you think that lxd is easier to use than Docker? I've always found docker build && docker push to be a simple interface to understand. What makes "start[ing] with lxd" more approachable or easier than creating a Dockerfile, given the abundance of Docker-101 tutorials, advice, and expertise available?
tombh 6y ago
I've always seen the story like this: It all started with Ruby. Ruby's syntactic sugar inspired the "syntactic sugar" of tooling, primarily Bundler and Rspec. Tooling, for what felt like the first time, became a first class citizen. Ruby's tooling made Heroku possible: ie, reproducible builds across; dev, testing, staging and production environments. Heroku's success was based on the primitives of the Twelve-Factor App[1]. The 12 factors (and therefore Heroku) were fundamentally designed around the already-old lightweight virtualisation technology of LXC. The success of Heroku paved the way for Docker. The success of Docker created the world in which Kubernetes makes sense. To be blunt: if you don't understand the relevance of Kubernetes, or whether it's relevant to you, you don't understand the benefits of the 12 factors in their broadest sense. The 12 factors are much, much more than just "How To Deploy On Heroku". Copypasting the 12 factors: I. Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages VI. Processes Execute the app as one or more stateless processes VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown X. Dev/prod parity Keep development, staging, and production as similar as possible XI. Logs Treat logs as event streams XII. Admin processes Run admin/management tasks as one-off processes 1. https://12factor.net/ https://12factor.net/
- LockAndLol 6y ago
  Reading this, I doubt it can be taken as gospel. > Store config in the environment How do you pass hierarchical config? In JSON, YAML, TOML, etc., it's easy to group env vars, but how do you do that with env vars? LOGGING__HANDLER__FORMAT, LOGGING__HANDLER__ARGS__ARG1, LOGGING__HANDLER__ARGS__ARG2 ? If so, that looks positively awful. If the solution is passing a YAML or JSON string in the env variable, that sounds even worse than having a config file that the app reads. >> it’s easy to mistakenly check in a config file to the repo Add it to the ignore file. Someone would really have to force commit it to get into version control. Plus, even if it's stored in env vars, you're going to commit the values somewhere be it your Ansible secrets, SaltStack yml, Chef or Puppet repo, whatever. To be fair, that's the only point I contend with. The rest is very reasonable.
deleted 6y ago
[deleted]
overgard 6y ago
I think one of the underrated parts of using something like Kubernetes early (or even w/ simpler orchestrators like swarm or rancher), is that it encourages (and sometimes enforces) architecture best practices from the start. IE, you won't be storing state locally, you'll be able to handle servers being randomly killed, you'll already have horizontal scaling, etc. In my experience the hard part of migrating to containers in a legacy app is when they break those constraints, especially around local state and special servers. It's easier to do these things sooner rather than later, and the constraints kubernetes places on you aren't that hard to work around if you design it in from the start. I good reason not to use kubernetes though is if you know your app is probably never going to scale, or if it's the kind of thing that can scale very well on a single machine, or if it's not primarily based on http communication. (I wouldn't write a real time game server in kubernetes, for instance, because I doubt it'd really help with a primarily UDP workload that is likely going to be attached to one server)
- throwaway894345 6y ago
  I’m not avidly opposed to k8s by any means, but you can get these same properties from any of a variety of easier-to-use schedulers such as Fargate, Heroku, or even EC2 autoscaling groups. Of course, there are probably Kubernetes distributions that lower the threshold of using Kubernetes (and if there aren’t, there really should be) by providing solutions for logging, monitoring, certificate management, Functions (a la AWS Lambda), load balancing / ingress, state management (databases as a service), etc preconfigured out of the box (similar to what you get with AWS or Heroku).
  - threeseed 6y ago
    The problem with your approach is that you're firmly locked in to the vendor. And in the case of Fargate, Heroku etc you're paying significantly more than if you had made use of Spot instances or shopped around for a cheaper vendor.
    - sukilot 6y ago
      Heroku costs about a nickel an hour per server, yeah? A dollar a day? So I'll worry about that price when I get to at least 10 servers
Axsuul 6y ago
I need Kubernetes since we're outgrowing Docker Swarm. Docker Swarm has a lot of issues we deal with on a constant basis so it's becoming quite painful. Can anyone suggest a good migration guide from Docker Swarm -> Kubernetes?
- ezekiel68 6y ago
  My gentle suggestion is to avoid phrases like "a lot of issues" and instead to list some of the top ones. This could give some people a chance to share how they have overcome some of those specific challenges. At a recent employer, we moved cold turkey from Swarm to Kubernetes (K8s in the rest of this reply). But we did this one microservice at a time. We didn't want our resulting K8s solution to be compromised by a misguided attempt to foist Swarm concept into the K8s way. Probably the biggest decision to make is whether to manage the cluster from scratch (not recommended), by using kops to deploy on a cloud platform, or use a cloud-native solution like EKS on AWS. After that -- here's a good guide to help with the differences in configuring the services[1] [1] https://kubernetes.io/docs/tasks/configure-pod-container/translate-compose-kubernetes/ https://kubernetes.io/docs/tasks/configure-pod-container/tra...
k__ 6y ago
This sounds like people try to compare K8s with what was before and sure, that all holds true. You get all of this with serverless (be it with managed services or FaaS), with reduced (albeit not zero) complexity compared to K8s.
- threeseed 6y ago
  And with serverless you get significant lock-in, are overpaying for resources, have less ability to debug when things go wrong, have basically zero flexibility and it's very difficult to have a local setup mirror your production one. It's fine for certain use cases but you get the benefits of both worlds by just using a managed K8s service like AWS EKS.
  - k__ 6y ago
    I'll dive into K8s this week, after a few years of serverless. I'm intrigued to see what it has to offer, that serverless lacks.
gatvol 6y ago
Probably not. Why not utilise ECS/ Fargate and attach managed services, rather then tending a whole now flock of things?
- gatvol 6y ago
  Using Docker-compose for local dev works very well for our teams, and we avoid the overhead of running and learning K8s operationally. That said this approach may change for us if we were large Health or insurance company scale, but we're not.
- aries1980 6y ago
  I find ECS/Fargate and managed Kubernetes not a bit easier to manage than Kubernetes itself. On the other hand, the inflexibility or vendor-specificity of these requires as much learning as you'd use FOSS tools. Also those skills are not transferable to an other cloud vendor, which might happened to be required if your primary cloud provider does not cover markets you supposed to operate in.
bdcravens 6y ago
Vendor lock-in aside, I've found ECS to be a great alternative.
- bg24 6y ago
  Yes if you are on AWS. Significantly speeds up the engineering workflow. Not sure if it is a vendor lock-in. If a customer wanted to move from ECS to kubernetes, they will need to migrate the manifests, roles/rolebinding etc. That is a small effort as long as they do not have to rewrite the code.
martythemaniak 6y ago
Remember Greenspun's 10th rule? "Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp." Well, there's a kubernetes version, which is that any sufficiently advanced deployment system will ultimately be half assed version of kubernetes.
- betaby 6y ago
  Including Kubernetes itself!
  - ezekiel68 6y ago
    To be fair, there was a ring of truth to this ...back in 2015 ;)
alexbanks 6y ago
No.
runxel 6y ago
Would have been much funnier, if this would have been just a page with a single word on it: NO. See also: http://dowebsitesneedtolookexactlythesameineverybrowser.com/ http://dowebsitesneedtolookexactlythesameineverybrowser.com/
kesor 6y ago
Probably not.
sytelus 6y ago
> Kubernetes isn’t just a 2018-era buzzword. And then he goes on to throw buzzwords. Yet another poorly written article as is now the norm for Kubetnetes.
- aliswe 6y ago
  Agreed, except the point about "the norm".
airnomad 6y ago
No.
enos_feedler 6y ago
I think what isn't fully appreciated about k8s yet, but we will look back on is how it creates an open standard platform for apps to be deployed. It is one thing to port your own apps to run within a k8s cluster, it is another to have and operate a k8s cluster that you can use to deploy services built by others. I hope we see more of this soon
deleted 6y ago
[deleted]
madushan1000 6y ago
I was a huge proponent of not hosting monolithic applications on kubernetes, that was until the company I worked for acquired another same size company and I had to learn their hand rolled puppet2/shell script based infrastructure management/deployment logic all over again(we had our own hand rolled puppet3/shell script based infrastructure logic too, so that's two stacks with their own quirks.) Now I'm completely in favor of hosting anything you can cram into kubernetes in kubernetes, even though kubernetes is more complex than most other infra tools, most of the time there is only one way to do things(configmaps for config, PV allocations for storage etc..) So if you understand kubernetes, it's easier to get the larger picture about the infrastructure even if you know nothing about the application stack.
ChicagoDave 6y ago
My assessment is that if you have a large, complex micro service environment with several applications, docker may be a good route to take as an interim to cloud native. If you have a relatively simple deployment structure (one big application with 10-20 services), maybe you don't need to add the docker skill to what you're already doing. If you're in the process of rebuilding your applications and you've decided cloud native is the way to go, then containers are pointless. I'd argue if you don't have the skills and you don't want to pay someone to do this work full-time, you might put your developers in a position to fail. Based on my 30+ years of building software, I see containers as a dead end. It may help you out in the short term, prove to your CTO that you're using "well-known" technology, but in the end, cloud native is going to replace everything. And before you say "but we can't be in the cloud," then you should know that cloud-native development like Lambdas can be done on-prem as well. I'm positive this post will get negative votes. That's fine. I like tilting at windmills.
- aliswe 6y ago
  Containers are a good tool for scaling resources?
  - ChicagoDave 6y ago
    It depends on the app. If it's an existing app, a container is fine. If you're building a new app and you don't care about the infrastructure, Serverless is better. My problem is that eventually all apps will be rewritten...and it's probable they will be in some kind of Serverless manner. Containers still have some separation of concerns issues as well as maintaining state or sharing data. My default would be Serverless with a plan B for Containers and a plan C for VM's.
devn0ll 6y ago
To me the question is more: Is there an extremely valid reason to _not_ use K8s. As a freelancer I visit quite a number of enterprise companies, think: Banks, Insurance, Airports, and they are all making the switch or are full-on invested in living in K8s by default. If it does not run in or was not made with K8s in mind: It will not be used/bought. Another thing I'm noticing with smaller companies: If you start fresh, you choose k8s. Which mens all other stuff is already slowly dying by virtue of not beeing chosen. Developers want/expect it, sysadmins see the benefits from day one, and companies see the potential gains of using less cloud resources and a platform that could potentially run in multiple clouds for the first time ever. K8s, openebs, prometheus/grafana, loki, kustomize, github actions. This is truly where "it's at" at the moment.
kissgyorgy 6y ago
I would say if you have the question "Do I need Kubernetes?", you don't need it because the benefits are not immediately crystal clear to you. Also, the author starts the evaluation from the wrong point of view, because the decision should be not so technical, but like everything else should be based on BUSINESS REQUIREMENTS. Evaluating Kubernetes is no different: - Do you have such high traffic that you need a distributed system? - Will a unified framework solve all your distribution problems? - Do you really need high availability? - Can you swallow the cost of high availability? - Can you handle the insane complexity of Kubernetes at a reasonable cost? You should not start asking questions about "Pods, Ingress" or anything Kubernetes specific, those are just implementation details.
- aliswe 6y ago
  > Can you handle the insane complexity of Kubernetes at a reasonable cost? For me, this point is akin to asking "Can you handle the insane complexity of Linux at a reasonable cost?" And what I mean by that is: If noone exists on your company who can administer it, then you shouldn't do it.
aliswe 6y ago
I've sprinkled some comments in this thread, but as someone who is working more or less full time with k8s infrastructure, architecture and maintenance (and I do love k8s!), my take is that if you have to ask this question then the answer is invariably: NO.
luord 6y ago
The article outlines a very nuanced detailing of how to answer that question, but I have a more blunt first consideration: If you need kubernetes, you don't need to ask whether you need it. Basically, I think that a team/product knows when the time has come in which the infrastructure has grown in complexity so much for it to need something like kubernetes to orchestrate it. If there are doubts, then whatever current setup is in place* is probably still enough and kubernetes is beyond what the team requires. I am very proud of the one time I managed to convince both my then tech lead and project manager, in one of my past jobs, to move away from kubernetes into a simpler architecture leveraging docker, compose and PaaS. * Hopefully one using docker and compose or similar, as mentioned in the article.

206 points by mocko 6y ago

Honestly nobody needs k8s, but nobody really needs anything. If you know it and you know it suits your needs, then sure! Go for it! Have fun. If you don't know it, then learn it first. Then you can properly evaluate whether it suits your requirements or not. There's huge value in choosing something that your team already knows + is familiar with I'd totally advocate for learning something new, but solving a business problem probably isn't the best time for this :) I think the article gave a great overview though, and is perhaps a good way to decide what to learn next

mrweasel 6y ago

I only have one tiny detail: Bursty traffic means that your cluster needs to be able to deal with the peaks. If you running an on prem Kubernetes cluster, then there is no savings, unless you can use the capacity for something else during non peak periods. The scaling, and potential savings is a cloud feature, not a feature of Kubernetes.

tstrimple 6y ago
It also enables you to achieve app density. Many of the companies I've been working with lately have large batch processes on nightly / weekly / monthly basis. For some reason, each job previously was setup on on-prem hardware dedicated per workload. Using Kubernetes and scheduling jobs to manage capacity has enabled us to reduce the number of on-prem servers substantially as part of the migration plan for the cloud.
- halbritt 6y ago
  Precisely. In my experience hosting services in VMs on-prem I was able to achieve roughly 30% efficiency across 3k instances and hundreds of nodes. In my experience hosting microservices on k8s, I was able to achieve 80% efficiency across hundreds of nodes. Both were the result of a great deal of work to optimize efficiency. In this case, I use the word "efficiency" to refer to a blend of CPU and memory utilization.
cat199 6y ago
> no savings This only relates to hardware cost - tooling impact can provide cost improvements if it enhances productivity (not always clear for k8s case)
aliswe 6y ago
I would nitpick that and say "if you are running a cluster with a static number of worker nodes". If I would have a cloud hosted k8s cluster and don't configure any auto scaling, the situation would be equivalent to me buying x number of physical machines and hosting them as worker nodes on prem. No ability to scale to "bursty traffic".
ec109685 6y ago
If the burst is for all apps at once, then yes. But if you need to handle temporary bursts of a single app among manh, then hosting them in a heterogenous cluster will give you benefit.

deleted 6y ago

[deleted]

ramraj07 6y ago

Not a single mention of elastic beanstalk or App Engine? The best middle ground for small teams who just want one reliable website with minimal scaling (and who can't just choose a nom-aws service).

rudolph9 6y ago
Do you have any examples of non-proprietary solutions?
- harpratap 6y ago
  Knative + buildpacks is one. CloudFoundry is open source too (although CF now runs on Kubernetes I think)
  - ramanujank 6y ago
    Cloud foundry offers their old(er) architecture as well as 2 ways to get the “cf push” experience over Kubernetes. Check out KubeCF and cf-for-k8s. Both of these are open source projects that you can deploy to a Kubernetes infrastructure of choice. [1] https://github.com/cloudfoundry-incubator/kubecf https://github.com/cloudfoundry-incubator/kubecf [2] https://github.com/cloudfoundry/cf-for-k8s https://github.com/cloudfoundry/cf-for-k8s
bpicolo 6y ago
I've found deployments to be much more slow than I'd hope for in both
cloudking 6y ago
We ran on AppEngine from launch to exit, great experience. Never had to worry about configurations, environments, maintenance, reliability, scalability etc. We just focused on building our product.
te_chris 6y ago
We tried to use app engine and it was a nightmare for us. K8s has a ramp, but it’s been incredibly smooth now that we’re setup and going.

gravypod 6y ago

> Run your applications in Docker from day 1 (with docker-compose it’s as valuable for dev as it is for production) and think carefully before letting your applications store local state. I think this is the key take away for many startups. Get it so you: 1. Have a single-command way to bring up your entire backend 2. That command can run on your dev machine 3. You document all of the settings for your containers and deps Once you have that in a docker-compose.yml file migrating to something like kube when you need health checks, autoscaling, etc is easy. The only thing you must be critical of is bashing people over the head to make everything you run in your company run in this environment you've created. No special deployments. Establish 1 method with 1 tool and go from there. Every company I've worked at I've brought a single command `docker-compose up -d` workflow and people were amazed by the productivity gains. No more remembering ports (container_name.local.company.com), no more chasing down configuration files and examples, no more talking to shared DBs, etc.

thom 6y ago
Can I ask people who _don't_ use Kubernetes and maybe have architectures built on proprietary cloud services: how do you manage this?
- gravypod 6y ago
  Depends on what cloud services you use. I am very opposed to using any cloud service that I can't self host in some way or another. That's a sure fire way to end up screwed. The main cloud services I use at work are: 1. S3/Azure Blob Store 2. DocumentDB/CosmoDB/Cloud Atlas 3. Aurora/A for PG DB 4. ElastiCache/some complex name in azure All of these data stores can be hosted locally using minio for s3, MongoDB for the mongo implementations, and postgres/mysql for the RDBMSs, redis/memcached for ElastiCache. If there was ever a case that I needed to ever use a real - full vendor lockin - solution I would hide it behind an API of my own and configure that API to store data locally. For example, the Azure Blob Store local container is completely broken for any sensible usage and I've been on an github issue thread for ~3 months waiting for them to resolve the issue (you're forced to link your netns to that container to talk to it as the application clients only works over 127.0.0.1). So essentially I have the following settings: - SERVICE_STORAGE_DRIVER=azure/fs - SERVICE_FS_LOCATION=/data - SERVICE_AZURE_CONTAINER=.... - SERVICE_AZURE_....=.... So in development it runs in fs mode, in prod it runs in azure mode, you mock the azure blob client in unit test to make sure that code works.

irontinkerer 6y ago

> To make a cluster useful for the average workload a menagerie of add-ons will be required. Some of them almost everyone uses, others are somewhat niche. This is the concern I have with k8s. All this complexity introduces operational and security concerns, while adding more work to do before you can just deploy business value (compared to launching on standard auto-scaling cloud instances)

orf 6y ago
Like most infrastructure work there is an upfront cost, but after paying that the result enables you to ship value faster and easier.
gravypod 6y ago
If you are using a managed kubernetes cluster from a cloud provider you mostly don't need to worry about these sorts of things. If you're not, and deploying to bare metal, the main things you need to worry about are: load balancers, storage & monitoring. If you're large enough that you can effectively run kube on bare metal you probably have enterprise solutions for load balacing [0], storage [1] & monitoring your applications that you've already validated as being secure/stable. If you want to go all out you can also grab an operator to manage rolling out databases for you (postgres [2], mongo, etc). A lot of the complexity people bump into with kube is really poorly planned out tools like Istio that have way too many features, a very overly complex mode of operation (out of the box it breaks CronJobs!!!), and very sub-standard community documentation. If you avoid Istio, or anything that injects sidecars and initcontainers, you'll find the experience enjoyable. [0] - https://clouddocs.f5.com/containers/v2/kubernetes/ https://clouddocs.f5.com/containers/v2/kubernetes/ [1] - https://www.netapp.com/us/kubernetes-storage.aspx https://www.netapp.com/us/kubernetes-storage.aspx [2] - https://github.com/CrunchyData/postgres-operator https://github.com/CrunchyData/postgres-operator
solatic 6y ago
As a rule of thumb, the quality of engineering of the core Kubernetes distribution is rock solid and incredible, but anything that's not is in varying stages of maturity. How much Kubernetes-adjacent code you actually need to adopt (and therefore, how much risk you take on) depends from project to project and organization to organization.

whotheffknows 6y ago

No.

peterwwillis 6y ago

Here's a quick reference: 1) Are you on AWS? Then you don't need Kubernetes. Use Fargate. 2) Are you on Google Cloud? Then you don't need Kubernetes. Use Cloud Run. 3) Are you on Azure? Then you don't need Kubernetes. Use Azure Container Instances. 4) Are you on a PaaS like Heroku? Then you don't need Kubernetes. 5) Are you on a random VPC provider / bare metal machines? You could probably still do without Kubernetes using Docker Swarm (apparently it's not dead!), Nomad, Mesos DC/OS, or a standard Linux box and systemd (or some other process or cluster scheduler). 6) Do you need to solve the bin-packing problem? Do you need to self-host a service mesh of microservices in multiple colocated regions? Do you need a fully automated redundant fault-tolerant network of disposable nodes to constantly reschedule different versions of applications with stringent RBACs, scheduled tasks, dynamic resource allocation, and do you have about a million dollars to spend on building and maintaining it all? Then you need Kubernetes.

jrott 6y ago
I get that feeling however I know for fargate there are some vertical scaling limitations that don't exist with kubernetes. Also at this point there is a ton of stuff out there that assumes you are deploying to kubernetes and makes that the path of least resistance.
- otterley 6y ago
  > I know for fargate there are some vertical scaling limitations that don't exist with kubernetes To overcome those limitations, you can use EC2 Container Instances with ECS. And instance auto scaling can now be automated for you via ECS Capacity Providers.
lazyant 6y ago
You may want to qualify these absolutes, as in "you may not need k8s, consider blah". For ex I run on k8s and can't use Fargate, same for GCP and Azure.
halbritt 6y ago
> do you have about a million dollars to spend on building and maintaining it all? Then you need Kubernetes. I think you're overstating the investment necessary to overcome the initial complication of Kubernetes and also understating the benefit of being on a platform with a massive and thriving community behind it. As an example, in a prior role, there were a set of data engineers that would receive data in the form of MS SQL server backups from which these engineers would need to query and transform data on an exploratory rather than production basis. Certainly one could use an "undifferentiated" service from a cloud provider, but it was also a roughly 5-minute process for me to use the rather high quality helm chart and docker image commonly available to stand up a new service for the benefit of each engineer that had the need. The process of creating that automation necessary to deploy the helm chart and restore the backups took approximately one hour and could be repeated ad nauseum in the aforementioned 5-minute time period. There are many, many other examples of this. Want a data-science platform complete out of the box with no vendor lock-in, how about data8.org? The list goes on.

dennis_jeeves 6y ago

kseistrup 6y ago
Exactly! Betteridge’s law of headlines: Any headline that ends in a question mark can be answered by the word no. ⌘ https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
- 2mol 6y ago
  I get it, but it's not really doing the author justice, who's trying to give a quite nuanced overview of where kubernetes is a good fit, and where it isn't. As somebody who's pretty skeptical about onboarding the big lump of complexity that is k8s, I really appreciated the information in the article.
  - dennis_jeeves 6y ago
    You see, any pile of junk(Kubernetes being one of them) can have a use case if you searched hard enough. Do not learn/work with Kubernetes unless your livelihood depends on it.
- dang 6y ago
  Mr. Betteridge retired after a thousand of these or so. https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=betteridge&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu.... Interestingly, it was pg who got it going on HN: https://news.ycombinator.com/item?id=4092880 https://news.ycombinator.com/item?id=4092880 I once used the joke about Mr. Betteridge retiring and got a reply pointing out that he hasn't: https://news.ycombinator.com/item?id=10393754 https://news.ycombinator.com/item?id=10393754.

kanobo 6y ago

When a newspaper headline or blog post title ends in a question mark, the answer is almost always 'no'.

dang 6y ago
https://news.ycombinator.com/item?id=24179969 https://news.ycombinator.com/item?id=24179969

spost 6y ago

I work for a startup whose product is small (half a dozen servers, if relatively beefy ones) clusters that will be run on-prem by customers, at least sometimes in a low-to-no-touch capacity. Most of our application components are micro-ish services that are run on all hosts in the cluster for either extra capacity or fault tolerance. We currently run everything on mesos/marathon, but are looking to switch away from it. K8s is kinda the “default” option, and is potentially appealing to some potential acquirers and investors. But I never really see k8s being talked about in that context of “physical hardware that’s on prem, but not on MY prem.” Is there a reason for that? If we go with k8s is it going to bite us? Does anyone have experience with something like that they could share?

dilyevsky 6y ago
You maybe interested in metal3.io And yes it will “bite” you but so will everything else.
lazyant 6y ago
I have some experience with this. Way I see it, if you have your software on somebody else's on-prem and move to k8s, more importantly than replacing the stack between Linux and your app is a change in mentality and demarcation of responsibilities, as in now your app "runs on k8s" and your clients are responsible for that layer (or they can contract that out) and you are responsible only for your app. It helps abstract out everything in the stack below your app and easier conceptually on everybody; now the clients can train in k8s and use same set of tooling like prometheus/grafana etc that usually go with k8s, same or similar RBAC access etc. OTOH realistic expectations need to be set; not because it's k8s there's going to be no problems or the learning/adaptation won't come without some pain. I suggest writing down some standard procedures for your clients like upgrades etc, pick same set of tooling for all of them (like same dashboard, same logging/monitoring/alerting etc) as a way to homeganize ("standardize") all of them. Feel free to email me.
solatic 6y ago
> But I never really see k8s being talked about in that context of “physical hardware that’s on prem, but not on MY prem.” Is there a reason for that? If we go with k8s is it going to bite us? Does anyone have experience with something like that they could share? Kubernetes provides a leaky abstraction above the underlying hardware - the storage and networking are going to be different depending on who is maintaining the Kubernetes cluster. Kubernetes's strength is that it acknowledges the leakiness of the abstraction and makes it explicit. If your customer uses a specific networking and storage provider, Kubernetes makes it easier for you to say (or not) that you have certified your product for those networking and storage providers, and here's what the manifests look like, because there's a standard way of configuring the application to work with that networking (CNI, which powers the standard Service as well as maybe NetworkPolicies) and storage (CSI, specifically StorageClass) provider. If you just provide Docker images, or VM appliances, then Murphy promises you that you're going to get frustrated support calls from customers saying "your application is slow and we don't understand why." Good luck then.

vasilakisfil 6y ago

For small apps and projects that continuous delivery is not required I would start with lxd. I think it's the easiest way to containerize an app. Once that feels too little, I would start looking at docker, and only when docker feels again too little, to kubernetes. So in essence, for 95% of the apps/people, the answer is no.

marcc 6y ago
Can you elaborate on why you think that lxd is easier to use than Docker? I've always found docker build && docker push to be a simple interface to understand. What makes "start[ing] with lxd" more approachable or easier than creating a Dockerfile, given the abundance of Docker-101 tutorials, advice, and expertise available?

tombh 6y ago

I've always seen the story like this: It all started with Ruby. Ruby's syntactic sugar inspired the "syntactic sugar" of tooling, primarily Bundler and Rspec. Tooling, for what felt like the first time, became a first class citizen. Ruby's tooling made Heroku possible: ie, reproducible builds across; dev, testing, staging and production environments. Heroku's success was based on the primitives of the Twelve-Factor App[1]. The 12 factors (and therefore Heroku) were fundamentally designed around the already-old lightweight virtualisation technology of LXC. The success of Heroku paved the way for Docker. The success of Docker created the world in which Kubernetes makes sense. To be blunt: if you don't understand the relevance of Kubernetes, or whether it's relevant to you, you don't understand the benefits of the 12 factors in their broadest sense. The 12 factors are much, much more than just "How To Deploy On Heroku". Copypasting the 12 factors: I. Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages VI. Processes Execute the app as one or more stateless processes VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown X. Dev/prod parity Keep development, staging, and production as similar as possible XI. Logs Treat logs as event streams XII. Admin processes Run admin/management tasks as one-off processes 1. https://12factor.net/ https://12factor.net/

LockAndLol 6y ago
Reading this, I doubt it can be taken as gospel. > Store config in the environment How do you pass hierarchical config? In JSON, YAML, TOML, etc., it's easy to group env vars, but how do you do that with env vars? LOGGING__HANDLER__FORMAT, LOGGING__HANDLER__ARGS__ARG1, LOGGING__HANDLER__ARGS__ARG2 ? If so, that looks positively awful. If the solution is passing a YAML or JSON string in the env variable, that sounds even worse than having a config file that the app reads. >> it’s easy to mistakenly check in a config file to the repo Add it to the ignore file. Someone would really have to force commit it to get into version control. Plus, even if it's stored in env vars, you're going to commit the values somewhere be it your Ansible secrets, SaltStack yml, Chef or Puppet repo, whatever. To be fair, that's the only point I contend with. The rest is very reasonable.

deleted 6y ago

[deleted]

overgard 6y ago

I think one of the underrated parts of using something like Kubernetes early (or even w/ simpler orchestrators like swarm or rancher), is that it encourages (and sometimes enforces) architecture best practices from the start. IE, you won't be storing state locally, you'll be able to handle servers being randomly killed, you'll already have horizontal scaling, etc. In my experience the hard part of migrating to containers in a legacy app is when they break those constraints, especially around local state and special servers. It's easier to do these things sooner rather than later, and the constraints kubernetes places on you aren't that hard to work around if you design it in from the start. I good reason not to use kubernetes though is if you know your app is probably never going to scale, or if it's the kind of thing that can scale very well on a single machine, or if it's not primarily based on http communication. (I wouldn't write a real time game server in kubernetes, for instance, because I doubt it'd really help with a primarily UDP workload that is likely going to be attached to one server)

throwaway894345 6y ago
I’m not avidly opposed to k8s by any means, but you can get these same properties from any of a variety of easier-to-use schedulers such as Fargate, Heroku, or even EC2 autoscaling groups. Of course, there are probably Kubernetes distributions that lower the threshold of using Kubernetes (and if there aren’t, there really should be) by providing solutions for logging, monitoring, certificate management, Functions (a la AWS Lambda), load balancing / ingress, state management (databases as a service), etc preconfigured out of the box (similar to what you get with AWS or Heroku).
- threeseed 6y ago
  The problem with your approach is that you're firmly locked in to the vendor. And in the case of Fargate, Heroku etc you're paying significantly more than if you had made use of Spot instances or shopped around for a cheaper vendor.
  - sukilot 6y ago
    Heroku costs about a nickel an hour per server, yeah? A dollar a day? So I'll worry about that price when I get to at least 10 servers

Axsuul 6y ago

I need Kubernetes since we're outgrowing Docker Swarm. Docker Swarm has a lot of issues we deal with on a constant basis so it's becoming quite painful. Can anyone suggest a good migration guide from Docker Swarm -> Kubernetes?

ezekiel68 6y ago
My gentle suggestion is to avoid phrases like "a lot of issues" and instead to list some of the top ones. This could give some people a chance to share how they have overcome some of those specific challenges. At a recent employer, we moved cold turkey from Swarm to Kubernetes (K8s in the rest of this reply). But we did this one microservice at a time. We didn't want our resulting K8s solution to be compromised by a misguided attempt to foist Swarm concept into the K8s way. Probably the biggest decision to make is whether to manage the cluster from scratch (not recommended), by using kops to deploy on a cloud platform, or use a cloud-native solution like EKS on AWS. After that -- here's a good guide to help with the differences in configuring the services[1] [1] https://kubernetes.io/docs/tasks/configure-pod-container/translate-compose-kubernetes/ https://kubernetes.io/docs/tasks/configure-pod-container/tra...

k__ 6y ago

This sounds like people try to compare K8s with what was before and sure, that all holds true. You get all of this with serverless (be it with managed services or FaaS), with reduced (albeit not zero) complexity compared to K8s.

threeseed 6y ago
And with serverless you get significant lock-in, are overpaying for resources, have less ability to debug when things go wrong, have basically zero flexibility and it's very difficult to have a local setup mirror your production one. It's fine for certain use cases but you get the benefits of both worlds by just using a managed K8s service like AWS EKS.
- k__ 6y ago
  I'll dive into K8s this week, after a few years of serverless. I'm intrigued to see what it has to offer, that serverless lacks.

gatvol 6y ago

Probably not. Why not utilise ECS/ Fargate and attach managed services, rather then tending a whole now flock of things?

gatvol 6y ago
Using Docker-compose for local dev works very well for our teams, and we avoid the overhead of running and learning K8s operationally. That said this approach may change for us if we were large Health or insurance company scale, but we're not.
aries1980 6y ago
I find ECS/Fargate and managed Kubernetes not a bit easier to manage than Kubernetes itself. On the other hand, the inflexibility or vendor-specificity of these requires as much learning as you'd use FOSS tools. Also those skills are not transferable to an other cloud vendor, which might happened to be required if your primary cloud provider does not cover markets you supposed to operate in.

bdcravens 6y ago

Vendor lock-in aside, I've found ECS to be a great alternative.

bg24 6y ago
Yes if you are on AWS. Significantly speeds up the engineering workflow. Not sure if it is a vendor lock-in. If a customer wanted to move from ECS to kubernetes, they will need to migrate the manifests, roles/rolebinding etc. That is a small effort as long as they do not have to rewrite the code.

martythemaniak 6y ago

Remember Greenspun's 10th rule? "Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp." Well, there's a kubernetes version, which is that any sufficiently advanced deployment system will ultimately be half assed version of kubernetes.

betaby 6y ago
Including Kubernetes itself!
- ezekiel68 6y ago
  To be fair, there was a ring of truth to this ...back in 2015 ;)

alexbanks 6y ago

No.

runxel 6y ago

Would have been much funnier, if this would have been just a page with a single word on it: NO. See also: http://dowebsitesneedtolookexactlythesameineverybrowser.com/ http://dowebsitesneedtolookexactlythesameineverybrowser.com/

kesor 6y ago

Probably not.

sytelus 6y ago

> Kubernetes isn’t just a 2018-era buzzword. And then he goes on to throw buzzwords. Yet another poorly written article as is now the norm for Kubetnetes.

aliswe 6y ago
Agreed, except the point about "the norm".

airnomad 6y ago

No.

enos_feedler 6y ago

I think what isn't fully appreciated about k8s yet, but we will look back on is how it creates an open standard platform for apps to be deployed. It is one thing to port your own apps to run within a k8s cluster, it is another to have and operate a k8s cluster that you can use to deploy services built by others. I hope we see more of this soon

deleted 6y ago

[deleted]

madushan1000 6y ago

I was a huge proponent of not hosting monolithic applications on kubernetes, that was until the company I worked for acquired another same size company and I had to learn their hand rolled puppet2/shell script based infrastructure management/deployment logic all over again(we had our own hand rolled puppet3/shell script based infrastructure logic too, so that's two stacks with their own quirks.) Now I'm completely in favor of hosting anything you can cram into kubernetes in kubernetes, even though kubernetes is more complex than most other infra tools, most of the time there is only one way to do things(configmaps for config, PV allocations for storage etc..) So if you understand kubernetes, it's easier to get the larger picture about the infrastructure even if you know nothing about the application stack.

ChicagoDave 6y ago

My assessment is that if you have a large, complex micro service environment with several applications, docker may be a good route to take as an interim to cloud native. If you have a relatively simple deployment structure (one big application with 10-20 services), maybe you don't need to add the docker skill to what you're already doing. If you're in the process of rebuilding your applications and you've decided cloud native is the way to go, then containers are pointless. I'd argue if you don't have the skills and you don't want to pay someone to do this work full-time, you might put your developers in a position to fail. Based on my 30+ years of building software, I see containers as a dead end. It may help you out in the short term, prove to your CTO that you're using "well-known" technology, but in the end, cloud native is going to replace everything. And before you say "but we can't be in the cloud," then you should know that cloud-native development like Lambdas can be done on-prem as well. I'm positive this post will get negative votes. That's fine. I like tilting at windmills.

aliswe 6y ago
Containers are a good tool for scaling resources?
- ChicagoDave 6y ago
  It depends on the app. If it's an existing app, a container is fine. If you're building a new app and you don't care about the infrastructure, Serverless is better. My problem is that eventually all apps will be rewritten...and it's probable they will be in some kind of Serverless manner. Containers still have some separation of concerns issues as well as maintaining state or sharing data. My default would be Serverless with a plan B for Containers and a plan C for VM's.

devn0ll 6y ago

To me the question is more: Is there an extremely valid reason to _not_ use K8s. As a freelancer I visit quite a number of enterprise companies, think: Banks, Insurance, Airports, and they are all making the switch or are full-on invested in living in K8s by default. If it does not run in or was not made with K8s in mind: It will not be used/bought. Another thing I'm noticing with smaller companies: If you start fresh, you choose k8s. Which mens all other stuff is already slowly dying by virtue of not beeing chosen. Developers want/expect it, sysadmins see the benefits from day one, and companies see the potential gains of using less cloud resources and a platform that could potentially run in multiple clouds for the first time ever. K8s, openebs, prometheus/grafana, loki, kustomize, github actions. This is truly where "it's at" at the moment.

kissgyorgy 6y ago

I would say if you have the question "Do I need Kubernetes?", you don't need it because the benefits are not immediately crystal clear to you. Also, the author starts the evaluation from the wrong point of view, because the decision should be not so technical, but like everything else should be based on BUSINESS REQUIREMENTS. Evaluating Kubernetes is no different: - Do you have such high traffic that you need a distributed system? - Will a unified framework solve all your distribution problems? - Do you really need high availability? - Can you swallow the cost of high availability? - Can you handle the insane complexity of Kubernetes at a reasonable cost? You should not start asking questions about "Pods, Ingress" or anything Kubernetes specific, those are just implementation details.

aliswe 6y ago
> Can you handle the insane complexity of Kubernetes at a reasonable cost? For me, this point is akin to asking "Can you handle the insane complexity of Linux at a reasonable cost?" And what I mean by that is: If noone exists on your company who can administer it, then you shouldn't do it.

aliswe 6y ago

I've sprinkled some comments in this thread, but as someone who is working more or less full time with k8s infrastructure, architecture and maintenance (and I do love k8s!), my take is that if you have to ask this question then the answer is invariably: NO.

luord 6y ago

The article outlines a very nuanced detailing of how to answer that question, but I have a more blunt first consideration: If you need kubernetes, you don't need to ask whether you need it. Basically, I think that a team/product knows when the time has come in which the infrastructure has grown in complexity so much for it to need something like kubernetes to orchestrate it. If there are doubts, then whatever current setup is in place* is probably still enough and kubernetes is beyond what the team requires. I am very proud of the one time I managed to convince both my then tech lead and project manager, in one of my past jobs, to move away from kubernetes into a simpler architecture leveraging docker, compose and PaaS. * Hopefully one using docker and compose or similar, as mentioned in the article.