4 ms·

> I use k8s at work, i know it has benefits, but it too often feels like using a bazooka to butter your toast. This is such a great description (I also for my

> I use k8s at work, i know it has benefits, but it too often feels like using a bazooka to butter your toast. This is such a great description (I also for my job work exclusively on a platform based on Kubernetes). I ran K8s at home successfully for a while using K3s (such a great project/tool) to become more proficient. After a few months I found that there were so many features I didn't need for a homelab that the complexity wasn't worth it. I feel like, at the moment, Kubernetes is awesome when you have two ingredients: 1. You go with a managed K8s offering (such as AWS EKS) 2. You have a team of engineers dedicated to the health of the K8s platform It's pretty cool that a somewhat small platform team can wield the scaleability of a company like Google, BUT they need to be very good with K8s. And many workloads probably don't need that much scaleability.

withinboredom 4y ago
For me, it’s more about simple redundancy. Prior to k8s, to reboot a node, you’d have to 1. Manually remove it from the load balancer(s) 2. Wait for connections to drain 3. Stop the services running so they can gracefully stop 4. Reboot the node 5. Do everything in reverse With k8s, all that can 100% be automated, basically for “free”.
- arcanemachiner 4y ago
  Ansible could do this pretty easily, no?
  - alex3305 4y ago
    Yes, with Ansible you would be able to set this up. But with Kubernetes it is basically free. Kubernetes will manage your workloads and if you set up your health checks correctly it will do this for you. Kubernetes is designed for such tasks. However. I would rather comfortably recommend Ansible for a home lab, but not Kubernetes. For something like that it is too complex with too many abstraction layers and too many caveats.
  - pas 4y ago
    the real "killer app" of "workload orchestration" is the active control loops, configured (somewhat naturally) declaratively. Ansible and other configuration managers and provisioning tools - as far as I know - had no way to do this. (they had policy conformance checks and whatnot, but fundamentally they did not make it easy to get to the desired good state, they ran on-demand and had to rely on ad-hoc information gathering, which made them slooooow and brittle)
- darkwater 4y ago
  it can be automated just like it was automatable before, you know have a REST API to do the automation, that's the big change wrt being on bare metal (but you had Puppet/Chef/Ansible for almost 15 years now). If you were in the cloud, you already had a REST API to do that before.
  - lyu07282 4y ago
    Its very different though, you don't tell it the steps to do, you tell it the world state, kubernetes figures out how to transition between changes between states and if something breaks it tries to fix it by going back to the state its supposed to be in. I think this immutable/indempotency aspect of kubernetes is a bit underappreciated, this can't possibly be compared with ansible.
    - darkwater 4y ago
      Puppet was/is declarative, or at least tried to be (unlike Ansible). Don't get me wrong, I completely understand tje differences with K8S and I'd rather accept "the inner loop" as the biggest differentiator.
    - withinboredom 4y ago
      Further, the workload /just moves/ to another node. In the old ways, the service was tied to the node it was on. If a disk failed, the service failed. Now, within moments, that service moves and the node can just go away. It’s almost magic. I think the reason k8s doesn’t make much sense in a home-lab setting is due to the limited nodes.
      - darkwater 4y ago
        The same magic that happens with autoscaling groups, if an instance or the app running on that instance die for any reason, it get recycled magically.
        withinboredom 4y ago

withinboredom 4y ago

For me, it’s more about simple redundancy. Prior to k8s, to reboot a node, you’d have to 1. Manually remove it from the load balancer(s) 2. Wait for connections to drain 3. Stop the services running so they can gracefully stop 4. Reboot the node 5. Do everything in reverse With k8s, all that can 100% be automated, basically for “free”.

arcanemachiner 4y ago
Ansible could do this pretty easily, no?
- alex3305 4y ago
  Yes, with Ansible you would be able to set this up. But with Kubernetes it is basically free. Kubernetes will manage your workloads and if you set up your health checks correctly it will do this for you. Kubernetes is designed for such tasks. However. I would rather comfortably recommend Ansible for a home lab, but not Kubernetes. For something like that it is too complex with too many abstraction layers and too many caveats.
- pas 4y ago
  the real "killer app" of "workload orchestration" is the active control loops, configured (somewhat naturally) declaratively. Ansible and other configuration managers and provisioning tools - as far as I know - had no way to do this. (they had policy conformance checks and whatnot, but fundamentally they did not make it easy to get to the desired good state, they ran on-demand and had to rely on ad-hoc information gathering, which made them slooooow and brittle)
darkwater 4y ago
it can be automated just like it was automatable before, you know have a REST API to do the automation, that's the big change wrt being on bare metal (but you had Puppet/Chef/Ansible for almost 15 years now). If you were in the cloud, you already had a REST API to do that before.
- lyu07282 4y ago
  Its very different though, you don't tell it the steps to do, you tell it the world state, kubernetes figures out how to transition between changes between states and if something breaks it tries to fix it by going back to the state its supposed to be in. I think this immutable/indempotency aspect of kubernetes is a bit underappreciated, this can't possibly be compared with ansible.
  - darkwater 4y ago
    Puppet was/is declarative, or at least tried to be (unlike Ansible). Don't get me wrong, I completely understand tje differences with K8S and I'd rather accept "the inner loop" as the biggest differentiator.
  - withinboredom 4y ago
    Further, the workload /just moves/ to another node. In the old ways, the service was tied to the node it was on. If a disk failed, the service failed. Now, within moments, that service moves and the node can just go away. It’s almost magic. I think the reason k8s doesn’t make much sense in a home-lab setting is due to the limited nodes.
    - darkwater 4y ago
      The same magic that happens with autoscaling groups, if an instance or the app running on that instance die for any reason, it get recycled magically.
      - withinboredom 4y ago