15 ms·

People love to pooh-pooh "complicated" things like unit tests, type systems, Kubernetes, GraphQL, etc. Things that are solving a specific problem for LARGE SCAL

People love to pooh-pooh "complicated" things like unit tests, type systems, Kubernetes, GraphQL, etc. Things that are solving a specific problem for LARGE SCALE ENTERPRISE users. I will quote myself here: A problem does not cease to exist just because you decided to ignore it. Without Kubernetes, you still need to: - Install software onto your machines - Start services - Configure your virtual machines to listen on specific ports - have a load balancer directing traffic to and watching the health of those ports - a system to re-start processes when they exit - something to take the logs of your systems and ship them to a centralized place so you can analyze them. - A place to store secrets and provide those secrets to your services. - A system to replace outdated services with newer versions ( for either security updates, or feature updates ). - A system to direct traffic to allow your services to communicate with one another. ( Service discovery ) - A way to add additional instances to a running service and tell the load balancer about them - A way to remove instances when they are no longer needed due to decreased load. So sure, you don't need Kubernetes at an enterprise organization! Just write all of that yourself! Great use of your time, instead of concentrating on writing features that will make your organization more money.

bajsejohannes 5y ago
> solving a specific problem The problem to me is that Kubernetes is not solving a specific problem, but a whole slew of problems. And some of them it's solving really poorly. For example, you can't really have downtime-free deploys in kubernetes (set a longish timer from SIGTERM to increase the chance that there's no downtime). Instead I'd rather solve each problem in a good way. It's not that hard. I'm not implementing it from scratch, but with good tools that exists outside of kubernetes and actually solve a specific problem.
- hnjst 5y ago
  You can actually renew/upgrade your whole cluster with no downtime if you care enough to tackle the annoying bits that cost a few minutes in YOLO mode.
- ferdowsi 5y ago
  Why can you not have downtime-free deploys? You tell your applications to drain connections and gracefully exit on SIGTERM. https://pkg.go.dev/net/http#Server.Shutdown https://pkg.go.dev/net/http#Server.Shutdown If your server is incapable of gracefully exiting, that's not a K8s problem.
  - afrodc_ 5y ago
    > Why can you not have downtime-free deploys? You tell your applications to drain connections and gracefully exit on SIGTERM. https://pkg.go.dev/net/http#Server.Shutdown https://pkg.go.dev/net/http#Server.Shutdown > If your server is incapable of gracefully exiting, that's not a K8s problem. Also whatever load balancer/service mesh you have can be configured for 503 rerouting within DC as necessary too.
  - bajsejohannes 5y ago
    > You tell your applications to drain connections and gracefully exit on SIGTERM. The problem is that k8s will send requests to your application after SIGTERM. So you have to wait some amount of time before shutting down to allow for that. This was at least the case last time I used k8s, and it seemed like it was due to the distributed architecture, so something that was more than a mere bugfix away.
- rhacker 5y ago
  K8s has like, probably the most complete support for readiness/no downtime deploys in the whole damn industry so it's surprising to hear that... https://cloud.google.com/blog/products/containers-kubernetes/kubernetes-best-practices-setting-up-health-checks-with-readiness-and-liveness-probes https://cloud.google.com/blog/products/containers-kubernetes...
exdsq 5y ago
You know, if there's one thing I've learnt from working in Tech, it's never ignore a pooh-pooh. I knew a Tech Lead who got pooh-poohed, made the mistake of ignoring the pooh-pooh. He pooh-poohed it! Fatal error! 'Cos it turned out all along that the developer who pooh-poohed him had been pooh-poohing a lot of other project managers who pooh-poohed their pooh-poohs. In the end, we had to disband the start-up. Morale totally destroyed... by pooh-pooh! Edit: Seeing as I'm already on negative karma people might not get the reference - https://www.youtube.com/watch?v=QeF1JO7Ki8E https://www.youtube.com/watch?v=QeF1JO7Ki8E
- AnIdiotOnTheNet 5y ago
  What makes you so sure that the downvotes aren't because all you posted was a comedic reference?
  - exdsq 5y ago
    I didn't know pooh-pooh was a genuine logical fallacy before now!
InternetPerson 5y ago
OK, true ... but if you do all that yourself, then "they" can never fire you, because no one else will know how the damn thing works. (Just be sure not to document anything!)
timr 5y ago
Sure, but k8s isn't the only way to do any of those things, and it's certainly a heavyweight way of doing most of them. It's not a question of k8s or bespoke. That's a false dichotomy. I see way too many young/inexperienced tech teams using k8s to build things that could probably be hosted on a couple of AWS instances (if that). The parasitic costs are high.
- rhacker 5y ago
  I see way too many young/inexperienced tech teams STILL using an unmaintainable process of just spinning up an EC2 instance for random crap because there is no deployment strategy at the company.
  - mountainriver 5y ago
    Yup, k8s is at least standardized in a way that’s somewhat sane. Before k8s every org I worked for had an absolute mess of tangled infrastructure
  - breakfastduck 5y ago
    Not sure why this is being downvoted. "We can do it ourselves!" attitude by people who are unskilled is the source of many legacy hell-webs sitting in companies all over the world that are desperately trying to be maintained by their inheritors.
  - timr 5y ago
    Not responsive to the argument. k8s is maybe a "deployment strategy", but it's certainly not the only one. Or the best one for all circumstances.
api 5y ago
It's not K8S or nothing. It's K8S or Nomad, which is a much simpler and easier to administrate solution.
- remram 5y ago
  While this is not false, I don't think many of the posts critical of K8s hitting the front page are advertising for Nomad, or focusing on drawbacks that don't apply to Nomad.
- rochacon 5y ago
  This is partially true. If the only feature you care about Kubernetes is container scheduling, then yes, Nomad is simpler. The same could probably be said about Docker Swarm. However, if you want service discovery, load balancing, secret management, etc., you'll probably need Nomad+Vault+Consul+Fabio/similar to get all the basic features. Want easy persistent storage provisioning? Add CSI to the mix. Configuring these services to work together is not all trivial either (considering proper security, such as TLS everywhere) and there aren't many solutions available from the community (or managed) that package this in an easy way.
jimbokun 5y ago
Yes, but you don't need many or any of those things to launch a Minimum Viable Product. So Kubernetes can become invaluable once you need to scale, but when you are getting started it will probably only slow you down.
- aequitas 5y ago
  If you want your MVP to be publicly available and your corporations ops/sec to be on board with your plans then Kubernetes is an answer as well. Even if your MVP only needs a single instance and no scaling. Kubernetes provides a common API between developers and operations so both can do the job they where hired for while being in each others way as least as possible.
  - jimbokun 5y ago
    Pre-MVP, development and ops are likely the same people.
    - aequitas 5y ago
      With Pre-MVP you mean installing it on your laptop right? It all just really depends on your companies size and the liberties you are given. At a certain size your company will have dedicated ops and security teams which call all the shots. For a lot of companies, Kubernetes gives developers the liberties they would normally only get with a lot of bureaucracy or red tape.
slackfan 5y ago
"Write it all yourself" - Install software onto your machines Package managers, thousands of them. - Start services SysVinit, and if shell is too complicated for you, you can write totally not-complicated unit files for SystemD. For most services, they already exist. - Configure your virtual machines to listen on specific ports Chef, Puppet, Ansible, other configuration tools, literally hundreds of them etc. - have a load balancer directing traffic to and watching the health of those ports Any commercial load balancer. - a system to re-start processes when they exit Any good init system will do this. - something to take the logs of your systems and ship them to a centralized place so you can analyze them. Syslog has had this functionality for decades. - A place to store secrets and provide those secrets to your services. A problem that is unique to kubernetes and serverless. Remember the days of assuming that your box was secure without having to do 10123 layers of abstraction? - A system to replace outdated services with newer versions ( for either security updates, or feature updates ). Package managers. - A system to direct traffic to allow your services to communicate with one another. ( Service discovery ) This is called an internal load balancer. - A way to add additional instances to a running service and tell the load balancer about them Most load balancers have built up processes for these. - A way to remove instances when they are no longer needed due to decreased load. maybe the only thing you may need to activelly configure, again in your load balancer. None of this really needs to be written itself, and these assumptions come from a very specific type of application architecture, which, no matter how much people try to make it, is not a one-size-fits-all solution.
- ferdowsi 5y ago
  I'm personally glad that Kubernetes has saved me from needing to manage all of this. I'm much more productive as an applications engineer now that I don't have to stare at a mountain of bespoke Ansible/Chef scripts operating on a Rube Goldberg machine of managed services.
  - cjalmeida 5y ago
    This x10. Each such setup is a unique snowflake of brittle Ansible/Bash scripts and unit files. Anything slightly different from the initial use case will break. Not to mention operations. K8s give you for free things that are a pain to setup otherwise. Want to autoscale your VMs based on load? Trivial in most cloud managed k8s.
sweeneyrod 5y ago
Unit tests and "type systems" have very little in common with Kubernetes and GraphQL.
- doctor_eval 5y ago
  Also, GraphQL is not complex. You can learn the basics in an hour or so.
NicoJuicy 5y ago
I have a Windows server and use dot net. I press right click - publish and for prod, i have to enter the password. Collecting logs uses the same mechanism as backups. They go to a cloud provider and are then easy to view by a web app. Never needed more for after hours than this, perhaps upgrading a server instance from running too many apps on 1 server.
alisonatwork 5y ago
That escalated quickly. Unit tests and type systems are not complicated at all, and are applied by solo developers all the time. GraphQL and Kubernetes are completely different beasts, technologies designed to solve problems that not all developers have. There really isn't a comparison to be made.
- doctor_eval 5y ago
  I agree. GraphQL is conceptually straightforward, even if certain implementations can be complex. Any developer familiar with static typing is going to get it pretty easily. I’m far from an expert, but ISTM that Kubernetes is complex both conceptually and in implementation. This has implications well beyond just operational reliability.
- strken 5y ago
  Almost every team I've worked on has needed to deploy multiple services somewhere, and almost every app has run into escalating round trip times from nested data and/or proliferating routes that present similar data in different ways. While it's true to say not all developers have those problems, they're very common.
  - alisonatwork 5y ago
    That's a very SaaS-centric way of looking at software development. Unit tests and type systems are useful across the whole stack. Systems developers, application developers, embedded developers, mobile developers, even sysadmins and IT people - they all have a use for these basic principles of how to design a piece of software. GraphQL and Kubernetes, on the other hand, are solutions designed exclusively for web services deployed into the cloud, and they're primarily useful in situations where there are many different teams each working on different services, with differing release schedules and engineering priorities. These situations might seem very common in large companies, but I don't think they represent common aspects of software development in general.
fxtentacle 5y ago
You're mixing together useful complexity with useless complexity. Plus at the very least, I'd be very careful about putting type systems into the same basket as Kubernetes. One is a basic language feature used offline and before deploying. The other is a highly complex interwoven web of tools that might take your systems offline if used incorrectly. Without Kubernetes, you need Debian and it's Apache and MySQL packages. It's called a LAMP stack and for many production deployments, that's good enough. Because without all that "cloud magic", a $50 per month sever running a bare metal OS is beyond overpowered for most web apps, so you can skip all the scaling exercises. And with a redundant PSU and a redundant network port, 99.99% uptime is achievable. A feat so difficult, I'd like to mention, that Amazon Web Services or Heruko rarely manage to... Complexity has high costs. Just because you don't see Kubernetes' complexity now, doesn't mean you won't pay for it through reduced performance, increased bug surface, increased downtime, or additional configuration nightmares.
- threeseed 5y ago
  This argument is often made and is ridiculous. No one should or is using Kubernetes to run a simple LAMP stack. But if you have dozens of containers and want them to be manager in a consistent, secure, observable and maintainable way then Kubernetes is going to be a better solution than anything you build yourself.
  - tmp_anon_22 5y ago
    > No one should or is using Kubernetes to run a simple LAMP stack. Yes they are. Some developer got all excited about the capabilities of k8s, and had an initial larger scope for a project, so they set it up with GKE or EKS, and it managed to provide just enough business value to barrow in like a tick that won't be going away for years. Developers get all excited for new shiny tools and chuck it into production all the time, particularly at smaller orgs.
    - threeseed 5y ago
      > initial larger scope for a project So they weren't trying to run a simple LAMP stack then.
lrdswrk00 5y ago
You have to do those things WITH Kubernetes. It doesn’t configure itself. I focused on fixing Kubernetes problems at my last job (usually networking). How is that supporting the business (hint: it didn’t so management forced us off Kubernetes) No piece of software is a panacea and shilling for project that’s intended to remind people Google exists, is not really putting time on anything useful either
deleted 5y ago
[deleted]
chubot 5y ago
(author here) The post isn't claiming that you should be doing something differently right now. It's claiming that there's something better that isn't discovered. Probably 10 years in the future. I will be really surprised if anyone really thinks that Kubernetes and even AWS is going to be the state of the art in 2031. (Good recent blog post and line of research I like about compositional cloud programming, from a totally different angle: https://medium.com/riselab/the-state-of-the-serverless-art-78a4f02951eb https://medium.com/riselab/the-state-of-the-serverless-art-7...) FWIW I worked with Borg for 8 years on many applications (and at Google for over a decade), so this isn't coming from nowhere. The author of the post I quoted worked with it even more: https://news.ycombinator.com/item?id=25243159 https://news.ycombinator.com/item?id=25243159 I was never an SRE, but I have written and deployed code to every data center at Google, as well as helping dozens of people like data scientists and machine learning researchers use it, etc. It's hard to use. I gave this post a modest title since I'm not doing anything about this right now, but I'm glad @genericlemon24 gave it some more visibility :)
- doctor_eval 5y ago
  This article really resonated with me. We are starting to run into container orchestration problems but I really don’t like what I read about K8s. Apart from anything else, it seems designed for much bigger problems than mine, and require the kind of huge mental effort to understand which, ironically, will make it harder for my business to grow. I’m curious if you’ve taken a look at Nomad and the other HashiCorp tools? They appear focussed and compositional, as you say, and this is why we are probably going to adopt them instead of K8s - they seem to be in a strong position to replace the core of K8s with something simpler.
  - carlosf 5y ago
    I use Nomad a lot in my company and I really like it. Our team tried to migrate to AWS ECS a few times and found it much harder to abstract stuff from devs / create self-service patterns. That said it's not a walk in the park. You will need to scratch your head a little bit to setup consul + nomad + vault + a load balancer correctly.
iammisc 5y ago
Comparing Kubernetes to type systems is like comparing a shack to a gothic cathedral. Type systems are incredibly stable. They have to be proved both sound and complete via meticulous argumentation. Once proven such, they work and their guarantees exist... no matter what. If you avoid the use of the `unsafe...` functions in languages like Haskell, you can be guaranteed of all the things the type system guarantees for you. In more structured languages like Idris or Coq, there is an absolute guarantee even on termination. This does not break. Whereas on kubernetes... things break all the time. There is no well-defined semantic model for how the thing works. This is a far cry from something like the calculus of inductive constructions (basis of COQ) for which there is a well-understood 'spec'. Anyone can implement COIC in their language if they understand the spec. You cannot say the same for kubernetes. Kubernetes is a nice bit of engineering. But it does not provide the same guarantees as type systems. In fact, of the four 'complicated' things you mentioned, only one thing has a well-defined semantic model and mathematically provable guarantees behind it. GraphQL is a particular language (and not one based on any great algebra either, like SQL), Kubernetes is just a program, and unit tests are just a technique. None of them are abstract entities with proven, unbreakable guarantees. Really, comparing Kubernetes to something like system FC or COIC is like comparing Microsoft Word to Stoke's theorem. The last thing I'll say is that type systems are incredibly easy. There are a few rules to memorize, but they are applied systematically. The same is not true of Kubernetes. Kubernetes breaks constantly. Its abstractions are incredibly leaky. It provides no guarantees other than an 'eventually'. And it is very complicated. There are myriad entities. Myriad operations. Myriad specs, working groups, etc. Type systems are relatively easy. There is a standard format for rules, and some proofs you don't really need to read through if you trust the experts.
ryanobjc 5y ago
Recently I had cause too try kubernetes… it has quite the rep so I gave myself an hour to see if I could get a simple container job running on it. I used GCP autopilot k8 cluster… and it was a slam dunk. I got it done in 30 minutes. I would highly recommend to others! And the cost is totally reasonable! Running a k8 cluster from scratch is def a bigco thing, but if you’re in the cloud then the solutions are awesome. Plus you can always move your workload elsewhere later if necessary.
- mountainriver 5y ago
  This is also my experience, k8s can get harder but for simple stuff it’s pretty dang easy
secondcoming 5y ago
We did all that on AWS, and do it now on GCE. Load balancers, instance groups, scaling policies, rolling updates... it's all automatic. If I wasn't on mobile I'd go into more detail. Config is ansible, jinja, blah blah the usual yaml mess.
wyager 5y ago
Comparing type systems to kubernetes seems like an incredible category error to me. They have essentially nothing in common except they both have something to do with computers. Also, there are plenty of well-designed and beautiful type systems, but k8s is neither of those.
fierro 5y ago
thank you lol. Hello World engineers will never stop criticizing K8s.
markzzerella 5y ago
Your post reads like a teenager yelling "you don't understand me" at parents who also were teenagers at one point. You really think that those are new and unique problems? Your bullet points are like a list of NixOS features. I just did all of that across half a dozen servers and a dozen virtual machines with `services.homelab.enable = true;` before I opened up HN while its deploying. I'm not surprised that you can't see us lowly peasants from your high horse but many of us have been doing everything you mentioned, probably far more reliably and reproducibly, for a long time.
- Aeolun 5y ago
  > Your post reads like a teenager yelling "you don't understand me" at parents who also were teenagers at one point. I don’t understand teenagers any more, and I’m barely 30. I don’t think this analogy really works. I agree with your point though.
  - markzzerella 5y ago
    You don't have to understand teenagers to understand that their problems are the same that they have always been, except in different settings.
Aeolun 5y ago
> Just write all of that yourself! At least then you’ll have a shot at actually understanding it. I can’t trust kubernetes when anything goes wrong because the system just isn’t very transparent.
skywhopper 5y ago
I’ve got a different experience with Kubernetes because from what I’ve seen in fails to provide most of the features you described out of the box. Or when it does, they have major issues of how to safely deploy and maintain them over time. I assume you mean all those services can be installed and configured on Kubernetes, once you have Kubernetes itself up and running. But that’s not the same thing.
kazinator 5y ago
If you don't write that yourself, you still have to understand how someone else wrote it so you can configure and use it properly, and understand how to debug it when it's not working.
GuB-42 5y ago
"Large scale enterprise" is the key here. Kubernetes was made by Google. Google is not your startup, it has millions of servers serving billions of users, of course it needs complex systems, and it has thousands of people to maintain them. In a small company, you probably don't need much of what's in that "need to" list. Rent a server, maybe a second one for redundancy, install your packages, run your app, and if you did things well, you can do quite a lot with a single machine. But a lot of people think they are Google, and get ready to scale to a level they will never reach, and do it badly. I think that where most of the pooh-poohing comes from, the use of overly complicated solutions for your scale.
- kaeshiwaza 5y ago
  It's where Thomson-Unix way win as KISS and still work for small to large scale.
amerkhalid 5y ago
My issue with Kubernetes and DevOps is companies that combine DevOps with development. As a developer, it is already hard enough to keep up with new frameworks. Now these companies want their devs to do DevOps, two vastly different expertise. Not sure how common it is in industry but I know enough developers who are now halfassing DevOps.
- hnjst 5y ago
  In my book DevOps is a set of practices that aims at improving the collaboration between Devs and Ops. I know that the term is now often used to label a role (or even a job description), but I think something important is lost in the switch. According to what I put behind the concept, implicating Devs is at the core. You built it, you run it!
  - amerkhalid 5y ago
    > In my book DevOps is a set of practices that aims at improving the collaboration between Devs and Ops. I think that's what sold me on DevOps. > You built it, you run it! This adds too much responsibilities for devs but also makes it hard to find good enough developers who can also manage deployments and infrastructure. I have never seen happy and competent Dev+DevOps person. There is just too much cognitive load for a same person to do these two things right at the same time. The Hello World of Kubernetes deployment is easy on Cloud but anytime you need to do something a bit more complex, learning curve increases tremendously. What seems to work is that each team having one or more dedicated DevOps person. Or I have seen a dedicated DevOps team in large orgs managing infrastructure for many other teams.
CTmystery 5y ago
Tangential to your point, but how did 'unit tests' end up in your list of complicated things? They are conceptually easy to understand, and they are certainly not only for large scale enterprise users. Granted, it takes years to learn how to write nice tests... maybe that is what you mean?
localhost 5y ago
Right. But that doesn’t mean that Kubernetes is the right solution for this set of problems.
o8r3oFTZPE 5y ago
"People like to ..." Perhaps the reason people question these complicated things is because they are, whether intentionally or not, being marketed to an audience on HN that includes small scale non-enterprise users. I shall paraphrase others here: A problem does not exist for you simply because it exists for LARGE SCALE ENTERPRISE users. What I would add to that is there is nothing particularly noteworthy about a large organisation's IT work simply because it is a large organisation or making billions in ad revenue, unless one is also working in a similar organisation. If some organisations are writing the next "Multics", it really should not be interesting to everyone. A single person who can do all the individual tasks you listed is likely to think critically when presented with "news" of organisations where no single individual can do those things. Its like how many Initech Corporation employees does it take to screw in a lightbulb. I find some of the most interesting work is found in projects started by individual programmers working alone. luajit for example.
fridif 5y ago
I do all of the above with my own jars I've written all by myself and I feel as if it was 10x faster than just scratching the surface of Kubernetes. Resume driven development is very real if people can't write their own load balancer.
jokethrowaway 5y ago
Even at smaller scale, dealing with any distro + k8s + helm can be simpler than doing ops on an bare ubuntu install. If your dependencies are not too many / well supported you may get good results with nix.
gfodor 5y ago
I like Chef Habitat. It pretty much does all this. It's how I've kept away from k8s this long.
deleted 5y ago
[deleted]
dikei 5y ago
Yep, we used to setup these things with a bunch of different systems using our collection of Ansible playbooks. The playbooks are complex, so as to handle all kind of edge cases. Furthermore, since they are developed over a long period, the coding convention are not uniformed; it's quite hard to teach the new hires how to use and contribute to the playbooks. We probably replaced tens of thousands line of Ansible code with a few thousands line of K8S code. We found the new code easier to maintain: because K8S is much stricter than Ansible, it's harder to deviate from the norm. Granted, we might be biased because K8S is all new and shiny, but so far we haven't regretted moving to K8S.
chillfox 5y ago
Not all of those things are actually needed at small to medium size.
o8r3oFTZPE 5y ago
"Poeple like to ..." Perhaps the reason people question these complicated things is because they are, whether intentionally or not, being marketed to an audience on HN that includes small scale non-enterprise users. I shall quote thyself here: A problem does not exist for you simply because it exists for LARGE SCALE ENTERPRISE users.
void_mint 5y ago
The Kubernetes marketing team has definitely gotten to you. The investment in DevRel is really paying off if people are unironically arguing that you _must_ use K8s or you're wasting money and time. I'd be very curious to find your proposed cost savings after accounting for those teams of engineers tasked with maintaining a company's K8s clusters. There is no free lunch.
MR4D 5y ago
> Things that are solving a specific problem for LARGE SCALE ENTERPRISE users. So, just like mainframes? /s Seriously, though, I think that was the point. The future needs to be a much less complicated tool.
dboreham 5y ago
Unit tests, GraphQL and type systems aren't complicated, or at least don't need to be.
_hoe_radi_22 5y ago
>> "complicated" things like unit tests, type systems, Kubernetes, GraphQL, etc. Those are not even in the same ballpark in terms of how complicated they are. Unit tests and type systems are not complicated at all. GraphQL not really either. But Kubernetes is very, very much.
phendrenad2 5y ago
Most people who are afraid of doing all of those things haven't actually done them, and would probably be shocked to find that they aren't actually that complex. Kubernetes actually makes them more complex, but simpler in the aggregate. In other words, most of the apparent value in kubernetes vanishes when you do a real bake-off between kubernetes vs rolling your own infrastructure. There is still SOME benefit, but it's usually exaggerated. Another problem with kubernetes is the flexibility it gives you. Look at five engineering teams using kubernetes, and you'll see five wildly different setups. Within that maneuverability, in a project that ostensibly makes things "simple", hides the devils that will bite you when you least expect it.
nikau 5y ago
The advantage of kubernetes is price and being a standard making staff hiring easier. F5, autosys, spunk, etc are all much better products to do the tasks you mentioned but cost $$$ vs kubernetes.
icythere 5y ago
Best luck finding some engineer can understand and do all those stuff today. It's possible, but it's hard. Everyone comes to the table with "Hey terraform and helm/k8s": D: D

bajsejohannes 5y ago

> solving a specific problem The problem to me is that Kubernetes is not solving a specific problem, but a whole slew of problems. And some of them it's solving really poorly. For example, you can't really have downtime-free deploys in kubernetes (set a longish timer from SIGTERM to increase the chance that there's no downtime). Instead I'd rather solve each problem in a good way. It's not that hard. I'm not implementing it from scratch, but with good tools that exists outside of kubernetes and actually solve a specific problem.

hnjst 5y ago
You can actually renew/upgrade your whole cluster with no downtime if you care enough to tackle the annoying bits that cost a few minutes in YOLO mode.
ferdowsi 5y ago
Why can you not have downtime-free deploys? You tell your applications to drain connections and gracefully exit on SIGTERM. https://pkg.go.dev/net/http#Server.Shutdown https://pkg.go.dev/net/http#Server.Shutdown If your server is incapable of gracefully exiting, that's not a K8s problem.
- afrodc_ 5y ago
  > Why can you not have downtime-free deploys? You tell your applications to drain connections and gracefully exit on SIGTERM. https://pkg.go.dev/net/http#Server.Shutdown https://pkg.go.dev/net/http#Server.Shutdown > If your server is incapable of gracefully exiting, that's not a K8s problem. Also whatever load balancer/service mesh you have can be configured for 503 rerouting within DC as necessary too.
- bajsejohannes 5y ago
  > You tell your applications to drain connections and gracefully exit on SIGTERM. The problem is that k8s will send requests to your application after SIGTERM. So you have to wait some amount of time before shutting down to allow for that. This was at least the case last time I used k8s, and it seemed like it was due to the distributed architecture, so something that was more than a mere bugfix away.
rhacker 5y ago
K8s has like, probably the most complete support for readiness/no downtime deploys in the whole damn industry so it's surprising to hear that... https://cloud.google.com/blog/products/containers-kubernetes/kubernetes-best-practices-setting-up-health-checks-with-readiness-and-liveness-probes https://cloud.google.com/blog/products/containers-kubernetes...

exdsq 5y ago

You know, if there's one thing I've learnt from working in Tech, it's never ignore a pooh-pooh. I knew a Tech Lead who got pooh-poohed, made the mistake of ignoring the pooh-pooh. He pooh-poohed it! Fatal error! 'Cos it turned out all along that the developer who pooh-poohed him had been pooh-poohing a lot of other project managers who pooh-poohed their pooh-poohs. In the end, we had to disband the start-up. Morale totally destroyed... by pooh-pooh! Edit: Seeing as I'm already on negative karma people might not get the reference - https://www.youtube.com/watch?v=QeF1JO7Ki8E https://www.youtube.com/watch?v=QeF1JO7Ki8E

AnIdiotOnTheNet 5y ago
What makes you so sure that the downvotes aren't because all you posted was a comedic reference?
- exdsq 5y ago
  I didn't know pooh-pooh was a genuine logical fallacy before now!

InternetPerson 5y ago

OK, true ... but if you do all that yourself, then "they" can never fire you, because no one else will know how the damn thing works. (Just be sure not to document anything!)

timr 5y ago

Sure, but k8s isn't the only way to do any of those things, and it's certainly a heavyweight way of doing most of them. It's not a question of k8s or bespoke. That's a false dichotomy. I see way too many young/inexperienced tech teams using k8s to build things that could probably be hosted on a couple of AWS instances (if that). The parasitic costs are high.

rhacker 5y ago
I see way too many young/inexperienced tech teams STILL using an unmaintainable process of just spinning up an EC2 instance for random crap because there is no deployment strategy at the company.
- mountainriver 5y ago
  Yup, k8s is at least standardized in a way that’s somewhat sane. Before k8s every org I worked for had an absolute mess of tangled infrastructure
- breakfastduck 5y ago
  Not sure why this is being downvoted. "We can do it ourselves!" attitude by people who are unskilled is the source of many legacy hell-webs sitting in companies all over the world that are desperately trying to be maintained by their inheritors.
- timr 5y ago
  Not responsive to the argument. k8s is maybe a "deployment strategy", but it's certainly not the only one. Or the best one for all circumstances.

api 5y ago

It's not K8S or nothing. It's K8S or Nomad, which is a much simpler and easier to administrate solution.

remram 5y ago
While this is not false, I don't think many of the posts critical of K8s hitting the front page are advertising for Nomad, or focusing on drawbacks that don't apply to Nomad.
rochacon 5y ago
This is partially true. If the only feature you care about Kubernetes is container scheduling, then yes, Nomad is simpler. The same could probably be said about Docker Swarm. However, if you want service discovery, load balancing, secret management, etc., you'll probably need Nomad+Vault+Consul+Fabio/similar to get all the basic features. Want easy persistent storage provisioning? Add CSI to the mix. Configuring these services to work together is not all trivial either (considering proper security, such as TLS everywhere) and there aren't many solutions available from the community (or managed) that package this in an easy way.

jimbokun 5y ago

Yes, but you don't need many or any of those things to launch a Minimum Viable Product. So Kubernetes can become invaluable once you need to scale, but when you are getting started it will probably only slow you down.

aequitas 5y ago
If you want your MVP to be publicly available and your corporations ops/sec to be on board with your plans then Kubernetes is an answer as well. Even if your MVP only needs a single instance and no scaling. Kubernetes provides a common API between developers and operations so both can do the job they where hired for while being in each others way as least as possible.
- jimbokun 5y ago
  Pre-MVP, development and ops are likely the same people.
  - aequitas 5y ago
    With Pre-MVP you mean installing it on your laptop right? It all just really depends on your companies size and the liberties you are given. At a certain size your company will have dedicated ops and security teams which call all the shots. For a lot of companies, Kubernetes gives developers the liberties they would normally only get with a lot of bureaucracy or red tape.

slackfan 5y ago

"Write it all yourself" - Install software onto your machines Package managers, thousands of them. - Start services SysVinit, and if shell is too complicated for you, you can write totally not-complicated unit files for SystemD. For most services, they already exist. - Configure your virtual machines to listen on specific ports Chef, Puppet, Ansible, other configuration tools, literally hundreds of them etc. - have a load balancer directing traffic to and watching the health of those ports Any commercial load balancer. - a system to re-start processes when they exit Any good init system will do this. - something to take the logs of your systems and ship them to a centralized place so you can analyze them. Syslog has had this functionality for decades. - A place to store secrets and provide those secrets to your services. A problem that is unique to kubernetes and serverless. Remember the days of assuming that your box was secure without having to do 10123 layers of abstraction? - A system to replace outdated services with newer versions ( for either security updates, or feature updates ). Package managers. - A system to direct traffic to allow your services to communicate with one another. ( Service discovery ) This is called an internal load balancer. - A way to add additional instances to a running service and tell the load balancer about them Most load balancers have built up processes for these. - A way to remove instances when they are no longer needed due to decreased load. maybe the only thing you may need to activelly configure, again in your load balancer. None of this really needs to be written itself, and these assumptions come from a very specific type of application architecture, which, no matter how much people try to make it, is not a one-size-fits-all solution.

ferdowsi 5y ago
I'm personally glad that Kubernetes has saved me from needing to manage all of this. I'm much more productive as an applications engineer now that I don't have to stare at a mountain of bespoke Ansible/Chef scripts operating on a Rube Goldberg machine of managed services.
- cjalmeida 5y ago
  This x10. Each such setup is a unique snowflake of brittle Ansible/Bash scripts and unit files. Anything slightly different from the initial use case will break. Not to mention operations. K8s give you for free things that are a pain to setup otherwise. Want to autoscale your VMs based on load? Trivial in most cloud managed k8s.

sweeneyrod 5y ago

Unit tests and "type systems" have very little in common with Kubernetes and GraphQL.

doctor_eval 5y ago
Also, GraphQL is not complex. You can learn the basics in an hour or so.

NicoJuicy 5y ago

I have a Windows server and use dot net. I press right click - publish and for prod, i have to enter the password. Collecting logs uses the same mechanism as backups. They go to a cloud provider and are then easy to view by a web app. Never needed more for after hours than this, perhaps upgrading a server instance from running too many apps on 1 server.

alisonatwork 5y ago

That escalated quickly. Unit tests and type systems are not complicated at all, and are applied by solo developers all the time. GraphQL and Kubernetes are completely different beasts, technologies designed to solve problems that not all developers have. There really isn't a comparison to be made.

doctor_eval 5y ago
I agree. GraphQL is conceptually straightforward, even if certain implementations can be complex. Any developer familiar with static typing is going to get it pretty easily. I’m far from an expert, but ISTM that Kubernetes is complex both conceptually and in implementation. This has implications well beyond just operational reliability.
strken 5y ago
Almost every team I've worked on has needed to deploy multiple services somewhere, and almost every app has run into escalating round trip times from nested data and/or proliferating routes that present similar data in different ways. While it's true to say not all developers have those problems, they're very common.
- alisonatwork 5y ago
  That's a very SaaS-centric way of looking at software development. Unit tests and type systems are useful across the whole stack. Systems developers, application developers, embedded developers, mobile developers, even sysadmins and IT people - they all have a use for these basic principles of how to design a piece of software. GraphQL and Kubernetes, on the other hand, are solutions designed exclusively for web services deployed into the cloud, and they're primarily useful in situations where there are many different teams each working on different services, with differing release schedules and engineering priorities. These situations might seem very common in large companies, but I don't think they represent common aspects of software development in general.

fxtentacle 5y ago

You're mixing together useful complexity with useless complexity. Plus at the very least, I'd be very careful about putting type systems into the same basket as Kubernetes. One is a basic language feature used offline and before deploying. The other is a highly complex interwoven web of tools that might take your systems offline if used incorrectly. Without Kubernetes, you need Debian and it's Apache and MySQL packages. It's called a LAMP stack and for many production deployments, that's good enough. Because without all that "cloud magic", a $50 per month sever running a bare metal OS is beyond overpowered for most web apps, so you can skip all the scaling exercises. And with a redundant PSU and a redundant network port, 99.99% uptime is achievable. A feat so difficult, I'd like to mention, that Amazon Web Services or Heruko rarely manage to... Complexity has high costs. Just because you don't see Kubernetes' complexity now, doesn't mean you won't pay for it through reduced performance, increased bug surface, increased downtime, or additional configuration nightmares.

threeseed 5y ago
This argument is often made and is ridiculous. No one should or is using Kubernetes to run a simple LAMP stack. But if you have dozens of containers and want them to be manager in a consistent, secure, observable and maintainable way then Kubernetes is going to be a better solution than anything you build yourself.
- tmp_anon_22 5y ago
  > No one should or is using Kubernetes to run a simple LAMP stack. Yes they are. Some developer got all excited about the capabilities of k8s, and had an initial larger scope for a project, so they set it up with GKE or EKS, and it managed to provide just enough business value to barrow in like a tick that won't be going away for years. Developers get all excited for new shiny tools and chuck it into production all the time, particularly at smaller orgs.
  - threeseed 5y ago
    > initial larger scope for a project So they weren't trying to run a simple LAMP stack then.

lrdswrk00 5y ago

You have to do those things WITH Kubernetes. It doesn’t configure itself. I focused on fixing Kubernetes problems at my last job (usually networking). How is that supporting the business (hint: it didn’t so management forced us off Kubernetes) No piece of software is a panacea and shilling for project that’s intended to remind people Google exists, is not really putting time on anything useful either

deleted 5y ago

[deleted]

chubot 5y ago

(author here) The post isn't claiming that you should be doing something differently right now. It's claiming that there's something better that isn't discovered. Probably 10 years in the future. I will be really surprised if anyone really thinks that Kubernetes and even AWS is going to be the state of the art in 2031. (Good recent blog post and line of research I like about compositional cloud programming, from a totally different angle: https://medium.com/riselab/the-state-of-the-serverless-art-78a4f02951eb https://medium.com/riselab/the-state-of-the-serverless-art-7...) FWIW I worked with Borg for 8 years on many applications (and at Google for over a decade), so this isn't coming from nowhere. The author of the post I quoted worked with it even more: https://news.ycombinator.com/item?id=25243159 https://news.ycombinator.com/item?id=25243159 I was never an SRE, but I have written and deployed code to every data center at Google, as well as helping dozens of people like data scientists and machine learning researchers use it, etc. It's hard to use. I gave this post a modest title since I'm not doing anything about this right now, but I'm glad @genericlemon24 gave it some more visibility :)

doctor_eval 5y ago
This article really resonated with me. We are starting to run into container orchestration problems but I really don’t like what I read about K8s. Apart from anything else, it seems designed for much bigger problems than mine, and require the kind of huge mental effort to understand which, ironically, will make it harder for my business to grow. I’m curious if you’ve taken a look at Nomad and the other HashiCorp tools? They appear focussed and compositional, as you say, and this is why we are probably going to adopt them instead of K8s - they seem to be in a strong position to replace the core of K8s with something simpler.
- carlosf 5y ago
  I use Nomad a lot in my company and I really like it. Our team tried to migrate to AWS ECS a few times and found it much harder to abstract stuff from devs / create self-service patterns. That said it's not a walk in the park. You will need to scratch your head a little bit to setup consul + nomad + vault + a load balancer correctly.

iammisc 5y ago

Comparing Kubernetes to type systems is like comparing a shack to a gothic cathedral. Type systems are incredibly stable. They have to be proved both sound and complete via meticulous argumentation. Once proven such, they work and their guarantees exist... no matter what. If you avoid the use of the `unsafe...` functions in languages like Haskell, you can be guaranteed of all the things the type system guarantees for you. In more structured languages like Idris or Coq, there is an absolute guarantee even on termination. This does not break. Whereas on kubernetes... things break all the time. There is no well-defined semantic model for how the thing works. This is a far cry from something like the calculus of inductive constructions (basis of COQ) for which there is a well-understood 'spec'. Anyone can implement COIC in their language if they understand the spec. You cannot say the same for kubernetes. Kubernetes is a nice bit of engineering. But it does not provide the same guarantees as type systems. In fact, of the four 'complicated' things you mentioned, only one thing has a well-defined semantic model and mathematically provable guarantees behind it. GraphQL is a particular language (and not one based on any great algebra either, like SQL), Kubernetes is just a program, and unit tests are just a technique. None of them are abstract entities with proven, unbreakable guarantees. Really, comparing Kubernetes to something like system FC or COIC is like comparing Microsoft Word to Stoke's theorem. The last thing I'll say is that type systems are incredibly easy. There are a few rules to memorize, but they are applied systematically. The same is not true of Kubernetes. Kubernetes breaks constantly. Its abstractions are incredibly leaky. It provides no guarantees other than an 'eventually'. And it is very complicated. There are myriad entities. Myriad operations. Myriad specs, working groups, etc. Type systems are relatively easy. There is a standard format for rules, and some proofs you don't really need to read through if you trust the experts.

ryanobjc 5y ago

Recently I had cause too try kubernetes… it has quite the rep so I gave myself an hour to see if I could get a simple container job running on it. I used GCP autopilot k8 cluster… and it was a slam dunk. I got it done in 30 minutes. I would highly recommend to others! And the cost is totally reasonable! Running a k8 cluster from scratch is def a bigco thing, but if you’re in the cloud then the solutions are awesome. Plus you can always move your workload elsewhere later if necessary.

mountainriver 5y ago
This is also my experience, k8s can get harder but for simple stuff it’s pretty dang easy

secondcoming 5y ago

We did all that on AWS, and do it now on GCE. Load balancers, instance groups, scaling policies, rolling updates... it's all automatic. If I wasn't on mobile I'd go into more detail. Config is ansible, jinja, blah blah the usual yaml mess.

wyager 5y ago

Comparing type systems to kubernetes seems like an incredible category error to me. They have essentially nothing in common except they both have something to do with computers. Also, there are plenty of well-designed and beautiful type systems, but k8s is neither of those.

fierro 5y ago

thank you lol. Hello World engineers will never stop criticizing K8s.

markzzerella 5y ago

Your post reads like a teenager yelling "you don't understand me" at parents who also were teenagers at one point. You really think that those are new and unique problems? Your bullet points are like a list of NixOS features. I just did all of that across half a dozen servers and a dozen virtual machines with `services.homelab.enable = true;` before I opened up HN while its deploying. I'm not surprised that you can't see us lowly peasants from your high horse but many of us have been doing everything you mentioned, probably far more reliably and reproducibly, for a long time.

Aeolun 5y ago
> Your post reads like a teenager yelling "you don't understand me" at parents who also were teenagers at one point. I don’t understand teenagers any more, and I’m barely 30. I don’t think this analogy really works. I agree with your point though.
- markzzerella 5y ago
  You don't have to understand teenagers to understand that their problems are the same that they have always been, except in different settings.

Aeolun 5y ago

> Just write all of that yourself! At least then you’ll have a shot at actually understanding it. I can’t trust kubernetes when anything goes wrong because the system just isn’t very transparent.

skywhopper 5y ago

I’ve got a different experience with Kubernetes because from what I’ve seen in fails to provide most of the features you described out of the box. Or when it does, they have major issues of how to safely deploy and maintain them over time. I assume you mean all those services can be installed and configured on Kubernetes, once you have Kubernetes itself up and running. But that’s not the same thing.

kazinator 5y ago

If you don't write that yourself, you still have to understand how someone else wrote it so you can configure and use it properly, and understand how to debug it when it's not working.

GuB-42 5y ago

"Large scale enterprise" is the key here. Kubernetes was made by Google. Google is not your startup, it has millions of servers serving billions of users, of course it needs complex systems, and it has thousands of people to maintain them. In a small company, you probably don't need much of what's in that "need to" list. Rent a server, maybe a second one for redundancy, install your packages, run your app, and if you did things well, you can do quite a lot with a single machine. But a lot of people think they are Google, and get ready to scale to a level they will never reach, and do it badly. I think that where most of the pooh-poohing comes from, the use of overly complicated solutions for your scale.

kaeshiwaza 5y ago
It's where Thomson-Unix way win as KISS and still work for small to large scale.

amerkhalid 5y ago

My issue with Kubernetes and DevOps is companies that combine DevOps with development. As a developer, it is already hard enough to keep up with new frameworks. Now these companies want their devs to do DevOps, two vastly different expertise. Not sure how common it is in industry but I know enough developers who are now halfassing DevOps.

hnjst 5y ago
In my book DevOps is a set of practices that aims at improving the collaboration between Devs and Ops. I know that the term is now often used to label a role (or even a job description), but I think something important is lost in the switch. According to what I put behind the concept, implicating Devs is at the core. You built it, you run it!
- amerkhalid 5y ago
  > In my book DevOps is a set of practices that aims at improving the collaboration between Devs and Ops. I think that's what sold me on DevOps. > You built it, you run it! This adds too much responsibilities for devs but also makes it hard to find good enough developers who can also manage deployments and infrastructure. I have never seen happy and competent Dev+DevOps person. There is just too much cognitive load for a same person to do these two things right at the same time. The Hello World of Kubernetes deployment is easy on Cloud but anytime you need to do something a bit more complex, learning curve increases tremendously. What seems to work is that each team having one or more dedicated DevOps person. Or I have seen a dedicated DevOps team in large orgs managing infrastructure for many other teams.

CTmystery 5y ago

Tangential to your point, but how did 'unit tests' end up in your list of complicated things? They are conceptually easy to understand, and they are certainly not only for large scale enterprise users. Granted, it takes years to learn how to write nice tests... maybe that is what you mean?

localhost 5y ago

Right. But that doesn’t mean that Kubernetes is the right solution for this set of problems.

o8r3oFTZPE 5y ago

"People like to ..." Perhaps the reason people question these complicated things is because they are, whether intentionally or not, being marketed to an audience on HN that includes small scale non-enterprise users. I shall paraphrase others here: A problem does not exist for you simply because it exists for LARGE SCALE ENTERPRISE users. What I would add to that is there is nothing particularly noteworthy about a large organisation's IT work simply because it is a large organisation or making billions in ad revenue, unless one is also working in a similar organisation. If some organisations are writing the next "Multics", it really should not be interesting to everyone. A single person who can do all the individual tasks you listed is likely to think critically when presented with "news" of organisations where no single individual can do those things. Its like how many Initech Corporation employees does it take to screw in a lightbulb. I find some of the most interesting work is found in projects started by individual programmers working alone. luajit for example.

fridif 5y ago

I do all of the above with my own jars I've written all by myself and I feel as if it was 10x faster than just scratching the surface of Kubernetes. Resume driven development is very real if people can't write their own load balancer.

jokethrowaway 5y ago

Even at smaller scale, dealing with any distro + k8s + helm can be simpler than doing ops on an bare ubuntu install. If your dependencies are not too many / well supported you may get good results with nix.

gfodor 5y ago

I like Chef Habitat. It pretty much does all this. It's how I've kept away from k8s this long.

deleted 5y ago

[deleted]

dikei 5y ago

Yep, we used to setup these things with a bunch of different systems using our collection of Ansible playbooks. The playbooks are complex, so as to handle all kind of edge cases. Furthermore, since they are developed over a long period, the coding convention are not uniformed; it's quite hard to teach the new hires how to use and contribute to the playbooks. We probably replaced tens of thousands line of Ansible code with a few thousands line of K8S code. We found the new code easier to maintain: because K8S is much stricter than Ansible, it's harder to deviate from the norm. Granted, we might be biased because K8S is all new and shiny, but so far we haven't regretted moving to K8S.

chillfox 5y ago

Not all of those things are actually needed at small to medium size.

o8r3oFTZPE 5y ago

"Poeple like to ..." Perhaps the reason people question these complicated things is because they are, whether intentionally or not, being marketed to an audience on HN that includes small scale non-enterprise users. I shall quote thyself here: A problem does not exist for you simply because it exists for LARGE SCALE ENTERPRISE users.

void_mint 5y ago

The Kubernetes marketing team has definitely gotten to you. The investment in DevRel is really paying off if people are unironically arguing that you _must_ use K8s or you're wasting money and time. I'd be very curious to find your proposed cost savings after accounting for those teams of engineers tasked with maintaining a company's K8s clusters. There is no free lunch.

MR4D 5y ago

> Things that are solving a specific problem for LARGE SCALE ENTERPRISE users. So, just like mainframes? /s Seriously, though, I think that was the point. The future needs to be a much less complicated tool.

dboreham 5y ago

Unit tests, GraphQL and type systems aren't complicated, or at least don't need to be.

_hoe_radi_22 5y ago

>> "complicated" things like unit tests, type systems, Kubernetes, GraphQL, etc. Those are not even in the same ballpark in terms of how complicated they are. Unit tests and type systems are not complicated at all. GraphQL not really either. But Kubernetes is very, very much.

phendrenad2 5y ago

Most people who are afraid of doing all of those things haven't actually done them, and would probably be shocked to find that they aren't actually that complex. Kubernetes actually makes them more complex, but simpler in the aggregate. In other words, most of the apparent value in kubernetes vanishes when you do a real bake-off between kubernetes vs rolling your own infrastructure. There is still SOME benefit, but it's usually exaggerated. Another problem with kubernetes is the flexibility it gives you. Look at five engineering teams using kubernetes, and you'll see five wildly different setups. Within that maneuverability, in a project that ostensibly makes things "simple", hides the devils that will bite you when you least expect it.

nikau 5y ago

The advantage of kubernetes is price and being a standard making staff hiring easier. F5, autosys, spunk, etc are all much better products to do the tasks you mentioned but cost $$$ vs kubernetes.

icythere 5y ago

Best luck finding some engineer can understand and do all those stuff today. It's possible, but it's hard. Everyone comes to the table with "Hey terraform and helm/k8s": D: D