4 ms·

> And your experience is irrelevant here... That seems unnecessarily harsh. But I do agree with your other points. Kubernetes introduces a lot of moving parts

by davb 8y ago

> And your experience is irrelevant here... That seems unnecessarily harsh. But I do agree with your other points. Kubernetes introduces a lot of moving parts, even if the minikube interface is relatively straightforward. Over the years, experience has taught me that black boxes don't stay opaque for long - something breaks and you ultimately have to learn the internals, usually in an emergency break/fix scenario. At FAANG, the gave teams who's specialism is running those sorts of orchestration systems. I trust systems I understand, and magic scares me.

pacala 8y ago
You do run Linux I suppose. Try to compile the Linux kernel from scratch and see for yourself the bewildering complexity hidden in there. Hey, there is a hypervirtualization module hidden in your kernel, with tens and tens of knobs to tweak if you are so inclined. And yet, 95% of that complexity is most likely unused by your workloads and whatever is left very very rarely surfaces during an emergency break scenario. When it does, it is indeed painful. I still remember pulling hair because of https://en.wikipedia.org/wiki/Nagle%27s_algorithm https://en.wikipedia.org/wiki/Nagle%27s_algorithm more than a decade ago. Conceptually Kubernetes is a trivial system, a Plan9 of orchestration systems. Everything is an executor watching a key/value store and triggering on various state configurations. What trips people is the network / DNS layer, which is also implemented as a bunch of executors watching the key/value store. When it breaks, you're helpless as a novice, plus debugging networks is painful for anyone. If Kubernetes had a '--network-driver=none' that just uses the host network as is, akin to Minikube & '--vm-driver=none', we'd never had this 'complexity' argument thrown around. Fortunately, once you understand the architecture, debugging the network is straightforward: poke around the system network/dns executor logs and the culprit will soon reveal itself.
- davb 8y ago
  > Try to compile the Linux kernel from scratch and see for yourself When I started using Linux around 20 years ago, with Red Hat 6, manually compiling the kernel was the only to way to enable all sorts of hardware support that's enabled by default today. And it was a great learning experience. Though I do get your point about complexity, and I agree that the kernel is (arguably necessarily) very complex. Unfortunately there are few mainstream, well-supported and broadly compatible alternatives. I miss the days when computers were simple and a single person could understand everything that goes on inside, all the way down to the hardware. With kubernetes and other container orchestration platforms, I feel that there are viable, less complex alternatives for many use cases. Simpler platforms mean fewer points of failure, and more chance that a small team of generalists would easily be able to fix any issues and get back to producing value for the business. There are definitely cases where kubernetes et al are warranted, but in order to provide any semblance of robustness would require a dedicated and well versed resource within the business to look after the system. And most businesses (I'd guess > 99%) don't have the scalability or reproducibility requirements to necessitate it. The extra layer of abstraction just isn't worth it for most teams.

by davb 8y ago

pacala 8y ago

You do run Linux I suppose. Try to compile the Linux kernel from scratch and see for yourself the bewildering complexity hidden in there. Hey, there is a hypervirtualization module hidden in your kernel, with tens and tens of knobs to tweak if you are so inclined. And yet, 95% of that complexity is most likely unused by your workloads and whatever is left very very rarely surfaces during an emergency break scenario. When it does, it is indeed painful. I still remember pulling hair because of https://en.wikipedia.org/wiki/Nagle%27s_algorithm https://en.wikipedia.org/wiki/Nagle%27s_algorithm more than a decade ago. Conceptually Kubernetes is a trivial system, a Plan9 of orchestration systems. Everything is an executor watching a key/value store and triggering on various state configurations. What trips people is the network / DNS layer, which is also implemented as a bunch of executors watching the key/value store. When it breaks, you're helpless as a novice, plus debugging networks is painful for anyone. If Kubernetes had a '--network-driver=none' that just uses the host network as is, akin to Minikube & '--vm-driver=none', we'd never had this 'complexity' argument thrown around. Fortunately, once you understand the architecture, debugging the network is straightforward: poke around the system network/dns executor logs and the culprit will soon reveal itself.

davb 8y ago
> Try to compile the Linux kernel from scratch and see for yourself When I started using Linux around 20 years ago, with Red Hat 6, manually compiling the kernel was the only to way to enable all sorts of hardware support that's enabled by default today. And it was a great learning experience. Though I do get your point about complexity, and I agree that the kernel is (arguably necessarily) very complex. Unfortunately there are few mainstream, well-supported and broadly compatible alternatives. I miss the days when computers were simple and a single person could understand everything that goes on inside, all the way down to the hardware. With kubernetes and other container orchestration platforms, I feel that there are viable, less complex alternatives for many use cases. Simpler platforms mean fewer points of failure, and more chance that a small team of generalists would easily be able to fix any issues and get back to producing value for the business. There are definitely cases where kubernetes et al are warranted, but in order to provide any semblance of robustness would require a dedicated and well versed resource within the business to look after the system. And most businesses (I'd guess > 99%) don't have the scalability or reproducibility requirements to necessitate it. The extra layer of abstraction just isn't worth it for most teams.