6 ms·

I think the single biggest mistake people make with Kubernetes is implementing it too soon. Last company I worked for spent piles of time fighting K8s when a si

I think the single biggest mistake people make with Kubernetes is implementing it too soon. Last company I worked for spent piles of time fighting K8s when a simple well implemented cluster would have done the job. It makes a lot of sense to build portable infrastructure, but you can scale a long ways with much simpler technologies.

sitkack 6y ago
Build debs, use the package manager. VMs are already an artificial abstraction, just use them.
- wahern 6y ago
  I would never voluntarily deploy anything using k8s or similar cloud stacks and services. But in practice the bulk of the complexity isn't in pushing and running the actual software, it's in provisioning attached resources (disks, databases), sharing certificates and keys, routing and load balancing, etc. Way too complex for my tastes, but if you're convinced you need a highly automated control plane for those things then pushing .deb or .rpm packages doesn't even come close to solving that problem. As usual it comes down to how you define the problem. You can't dissuade people from using k8s by comparing and contrasting k8s to alternatives; you've already ceded the debate over how to define the problem at that point. W'ever its relative merits, k8s is a reasonable approach to the problem of automating the control plane for "scaleable" services.
  - sitkack 6y ago
    Debs move state (and computation) and provide a way to resolve dependencies. Done right they also provide rollback. Debs can moved a whole lot more than just the software. Not saying it is the answer to everything, but it can do a whole lot more than software. Most of our problems are self made we can unmake them by changing the rules we operate under. > As usual it comes down to how you define the problem. Totally. Redefine the problem until the solution is tractable and simple. K8s is the problem to a solution.
- ex_amazon_sde 6y ago
  Ex Amazon SDE here. You are absolutely right.
otabdeveloper4 6y ago
I've only played with k8s for like an hour in total, but it's pretty obvious to me that k8s fell very much victim to the second system effect. It's an insanely complex solution to a very niche problem - scaling stateless web app backend nodes written in scripting languages. Stray even a little bit off the garden path and you start feeling pain.
- ec109685 6y ago
  You only played with it an hour and you reached all these conclusions?
  - otabdeveloper4 6y ago
    Yes, because I saw that my (not-so-niche, actually) use case wasn't even considered, and "noped" outta there as fast as I could.
    - 3131s 6y ago
      What was your use case?
      - otabdeveloper4 6y ago
        Some ETL logic packaged in Docker containers with a bit of simple scheduling and orchestration. K8s sounds like a good idea on paper - you get reproducibility and resilence "for free" - but then I found out I have to effectively roll my own everything with k8s anyways and went with Jenkins instead.
- cheriot 6y ago
  > scaling stateless web app backend nodes written in scripting languages K8s solves a far wider problem space. Need to run a data store? Use a StatefulSet and PersistentVolumes. Need an occasional task? Jobs and CronJobs. Need to know what's happening? Metrics and logs have APIs. Load balancing? Ingress? Firewalls? Security? If someone knew nothing of operating systems other than a class on MINIX, I suspect running a massive datacenter would be easier with k8s than running a medium system of debian boxes.
sa46 6y ago
I hear this refrain consistently on HN but of the options I've tried, GKE is the most pleasant for my single person app. - App Engine has a bunch of weird limitations and slow deploy times. Qualitatively, it feels like the spotlight has moved on. - Running my own compute instances felt like reinventing Kubernetes, especially once you roll your own deploy mechanism and throw load balancing in the mix. I also don't buy that it's easier. - Cloud run is promising but for database heavy apps it's a non-starter. - GKE was pretty smooth. It feels like it gets a lot more love than App Engine. The UI was functional with lots of depth. Once I push a docker image, GKE updates the nodes to serve the latest version. Load balancing was a matter of ~3 yaml files at 10 lines a pop.
- ogre_codes 6y ago
  You are mistaking my point here. I'm not suggesting you replace K8s with some other thing which does the same things k8s does, but that you build out a simpler/ easier to maintain solution until you need something like k8s. Many Many Many businesses can do just fine without the kind of complexity K8s (or App Engine or GKE) brings to the table. There are a lot of businesses that will never have to deal with dynamic scaling, engineering autoscaling into those solutions is pointless. If you need that kind of scaling then K8r is fantastic. My point is a lot of people turn to K8s well before they need to or without understanding why they might need it.
collyw 6y ago
My place has Kubernetes for when we are ready to scale. I am not sure it has gone above the default two pods. It did cause problems with moderately large uploads (5gb), as the memory gets chewed up far faster when using multiple small machines versus one big machine. I am not sure we will ever actually need this level of scaling.
deleted 6y ago
[deleted]

sitkack 6y ago

Build debs, use the package manager. VMs are already an artificial abstraction, just use them.

wahern 6y ago
I would never voluntarily deploy anything using k8s or similar cloud stacks and services. But in practice the bulk of the complexity isn't in pushing and running the actual software, it's in provisioning attached resources (disks, databases), sharing certificates and keys, routing and load balancing, etc. Way too complex for my tastes, but if you're convinced you need a highly automated control plane for those things then pushing .deb or .rpm packages doesn't even come close to solving that problem. As usual it comes down to how you define the problem. You can't dissuade people from using k8s by comparing and contrasting k8s to alternatives; you've already ceded the debate over how to define the problem at that point. W'ever its relative merits, k8s is a reasonable approach to the problem of automating the control plane for "scaleable" services.
- sitkack 6y ago
  Debs move state (and computation) and provide a way to resolve dependencies. Done right they also provide rollback. Debs can moved a whole lot more than just the software. Not saying it is the answer to everything, but it can do a whole lot more than software. Most of our problems are self made we can unmake them by changing the rules we operate under. > As usual it comes down to how you define the problem. Totally. Redefine the problem until the solution is tractable and simple. K8s is the problem to a solution.
ex_amazon_sde 6y ago
Ex Amazon SDE here. You are absolutely right.

otabdeveloper4 6y ago

I've only played with k8s for like an hour in total, but it's pretty obvious to me that k8s fell very much victim to the second system effect. It's an insanely complex solution to a very niche problem - scaling stateless web app backend nodes written in scripting languages. Stray even a little bit off the garden path and you start feeling pain.

ec109685 6y ago
You only played with it an hour and you reached all these conclusions?
- otabdeveloper4 6y ago
  Yes, because I saw that my (not-so-niche, actually) use case wasn't even considered, and "noped" outta there as fast as I could.
  - 3131s 6y ago
    What was your use case?
    - otabdeveloper4 6y ago
      Some ETL logic packaged in Docker containers with a bit of simple scheduling and orchestration. K8s sounds like a good idea on paper - you get reproducibility and resilence "for free" - but then I found out I have to effectively roll my own everything with k8s anyways and went with Jenkins instead.
cheriot 6y ago
> scaling stateless web app backend nodes written in scripting languages K8s solves a far wider problem space. Need to run a data store? Use a StatefulSet and PersistentVolumes. Need an occasional task? Jobs and CronJobs. Need to know what's happening? Metrics and logs have APIs. Load balancing? Ingress? Firewalls? Security? If someone knew nothing of operating systems other than a class on MINIX, I suspect running a massive datacenter would be easier with k8s than running a medium system of debian boxes.

sa46 6y ago

I hear this refrain consistently on HN but of the options I've tried, GKE is the most pleasant for my single person app. - App Engine has a bunch of weird limitations and slow deploy times. Qualitatively, it feels like the spotlight has moved on. - Running my own compute instances felt like reinventing Kubernetes, especially once you roll your own deploy mechanism and throw load balancing in the mix. I also don't buy that it's easier. - Cloud run is promising but for database heavy apps it's a non-starter. - GKE was pretty smooth. It feels like it gets a lot more love than App Engine. The UI was functional with lots of depth. Once I push a docker image, GKE updates the nodes to serve the latest version. Load balancing was a matter of ~3 yaml files at 10 lines a pop.

ogre_codes 6y ago
You are mistaking my point here. I'm not suggesting you replace K8s with some other thing which does the same things k8s does, but that you build out a simpler/ easier to maintain solution until you need something like k8s. Many Many Many businesses can do just fine without the kind of complexity K8s (or App Engine or GKE) brings to the table. There are a lot of businesses that will never have to deal with dynamic scaling, engineering autoscaling into those solutions is pointless. If you need that kind of scaling then K8r is fantastic. My point is a lot of people turn to K8s well before they need to or without understanding why they might need it.

collyw 6y ago

My place has Kubernetes for when we are ready to scale. I am not sure it has gone above the default two pods. It did cause problems with moderately large uploads (5gb), as the memory gets chewed up far faster when using multiple small machines versus one big machine. I am not sure we will ever actually need this level of scaling.

deleted 6y ago

[deleted]