3 ms·

Really love seeing such a straightforward example, of starting with some desire or need, and ending up DIY'ing ones own operator. A lot of the pushback against

Really love seeing such a straightforward example, of starting with some desire or need, and ending up DIY'ing ones own operator. A lot of the pushback against Kubernetes revolves around whether you 'rewlly need it' or whether to do something else. Seeing someone go past running containers like this highlights the extensibility, shows the core of Kubernetes as a pattern & paradigm for building any kind of platform. It's neat seeing that done so quickly & readily here. That we can add and manage anything, consistently, quickly, is a promise that I love seeing fulfilled.

Etheryte 2y ago
While I see what you're getting at, I find the comment funny after the first thing the article leads with is that certain things are a pain in Kubernetes.
- beeboobaa3 2y ago
  It's a pain because kubernetes is designed to run multiple workloads on multiple servers. So if you want to access the VPN from some kubernetes containers you're going to have to figure something out. But nothing is stopping you from just joining all your hosts into the VPN, just like a traditional deployment. Or set it up on your network gateway. This would make it available to all of your containers. Great. You're done. But if that's not what you want, you'll need to figure something out.
  - Etheryte 2y ago
    I'm not sure if I follow. You said Kubernetes is great for building any kind of a platform, but then when someone wants controlled access to a VPN it suddenly turns into a no, not like that? Giving only certain parts of your architecture access to certain capabilities is far from a niche use case.
    - themgt 2y ago
      Giving only certain parts of your architecture access to certain capabilities is far from a niche use case What is the "normal" best practice here then? I would just spin up multiple single-node k3s VM clusters and hook the AI k3s VM to the VPN and the others not.
    - beeboobaa3 2y ago
      > You said Kubernetes is great for building any kind of a platform I didn't, that was someone else. I wouldn't say any kind of platform, but it is a great foundation for many platforms. > it suddenly turns into a no, not like that Not at all. Read my post and the OP again! Several valid solutions have been offered, some that work out of the box, some that require a little tinkering. If you want to do a traditional VM deployment you'd segment your workloads per hosts and put some hosts in the VPN. Sure. Cool. You can do exactly the same with kubernetes with node pools. Only when you want to have some workloads that have access and some that don't running on the same host machine you might need to do some tinkering. Just like you'd have to do otherwise. Kubernetes really changes nothing, other than giving you ways to deal with it. Of course there's plenty of ready built solutions for this you can just plug in, too. Search for Service Mesh.
- nonameiguess 2y ago
  Sort of. What the author is trying to do is actually quite easy if the entire setup had been Kubernetes. k3s allows you to connect nodes to your cluster over a Tailscale VPN. If you do that, you don't need to expose your remote AI server to any network at all except the Kubernetes internal network. I'm guessing fly.io doesn't just give you bare servers you can run k3s on, though. The only real difficulty here is hooking up totally different abstraction engines that aren't designed to work with each other. Putting the blame specifically on one of them and not the other doesn't make sense.
- jauntywundrkind 2y ago
  The pluggable storage (CSI) and networking (CNI) aspects of Kube tend to be a little limiting, in that a cluster generally only expects to have one at a time running. There is Multus, which is extremely well regarded CNI that enables mixing and matching different CNI Plugins. It works great, but it does involve diving into an additional level of complexity, and it can be exciting/unstable building the setup. Maybe the author didn't have rights over the cluster to do this or interest in mucking about deeply in this admittedly complex/highly capable system was limited. Or maybe just using an adding like this was appealing as non disruptive! This seems like a pretty creative & direct way to extend what they already had. There are good well tread options for higher power, for mixed modes of networking, but this strikes me as a nice way to add more without having to revamp the base cluster, which I found super cool & direct. https://github.com/k8snetworkplumbingwg/multus-cni https://github.com/k8snetworkplumbingwg/multus-cni
candiddevmike 2y ago
At its heart, Kubernetes is a workload scheduler, and a fairly opinionated one at that. IMO, it's too complicated for folks that just need an "extensible platform" when all they really need is a dumb scheduler API without all the Kubernetes bells and whistles that have been bolted on over the years. Kubernetes tries to be everything to everyone and makes the entire thing too complicated. Seeing Kubernetes broken up into smaller, more purpose built components and let folks pick and choose or swap could be helpful, at the risk of it becoming OpenStack. There's an alternate reality where Fleet is the defacto scheduler and cats and dogs live together in harmony.
- mrgaro 2y ago
  I'd say the opposite instead: we need Kubernetes distributions, just like Linux needs distributions. Nobody wants to build their kernel from scratch and to hand pick various user space programs. Same for Kubernetes: Distributions which pack everything you need in an opinionated way, so that it's easy to use. Now it's kinda build-your-own-kubernetes at every platform: kubeadm, EKS etc all require you to install various add-on components before you have a fully suitable cluster.
  - p-o 2y ago
    I think the Operator pattern will grow to become exactly what you describe. We're still at the early stage of that, but I can see that a group of operators could become a "distribution", in your example.
  - jen20 2y ago
    There have been distributions of Kubernetes for almost as long as there has been Kubernetes.
- koito17 2y ago
  > At its heart, Kubernetes is a workload scheduler Wouldn't it be more accurate to state Kubernetes at its core is a distributed control plane? You want something that inspects the state of a distributed system and mutates it as necessary to reach some "desired state". That's the control plane bit. For fault tolerance, the control plane itself needs to be distributed. This is the exact reason for Kubernetes' existence.
treflop 2y ago
What OP did could be done with any containerization platform that abstracted away networking, which should be all of them. Personally I think you should always be using containerization, because even single node Docker is easy. If you are running something for real, then definitely use Kubernetes. If you are using containerization, setting up Tailscale is trivial. Whenever you abstract away something, you can swap out core components like networking willy nilly. Of course, you can always over-abstract, but a healthy amount is wonderful for most non-very high performance use cases.

Etheryte 2y ago

While I see what you're getting at, I find the comment funny after the first thing the article leads with is that certain things are a pain in Kubernetes.

beeboobaa3 2y ago
It's a pain because kubernetes is designed to run multiple workloads on multiple servers. So if you want to access the VPN from some kubernetes containers you're going to have to figure something out. But nothing is stopping you from just joining all your hosts into the VPN, just like a traditional deployment. Or set it up on your network gateway. This would make it available to all of your containers. Great. You're done. But if that's not what you want, you'll need to figure something out.
- Etheryte 2y ago
  I'm not sure if I follow. You said Kubernetes is great for building any kind of a platform, but then when someone wants controlled access to a VPN it suddenly turns into a no, not like that? Giving only certain parts of your architecture access to certain capabilities is far from a niche use case.
  - themgt 2y ago
    Giving only certain parts of your architecture access to certain capabilities is far from a niche use case What is the "normal" best practice here then? I would just spin up multiple single-node k3s VM clusters and hook the AI k3s VM to the VPN and the others not.
  - beeboobaa3 2y ago
    > You said Kubernetes is great for building any kind of a platform I didn't, that was someone else. I wouldn't say any kind of platform, but it is a great foundation for many platforms. > it suddenly turns into a no, not like that Not at all. Read my post and the OP again! Several valid solutions have been offered, some that work out of the box, some that require a little tinkering. If you want to do a traditional VM deployment you'd segment your workloads per hosts and put some hosts in the VPN. Sure. Cool. You can do exactly the same with kubernetes with node pools. Only when you want to have some workloads that have access and some that don't running on the same host machine you might need to do some tinkering. Just like you'd have to do otherwise. Kubernetes really changes nothing, other than giving you ways to deal with it. Of course there's plenty of ready built solutions for this you can just plug in, too. Search for Service Mesh.
nonameiguess 2y ago
Sort of. What the author is trying to do is actually quite easy if the entire setup had been Kubernetes. k3s allows you to connect nodes to your cluster over a Tailscale VPN. If you do that, you don't need to expose your remote AI server to any network at all except the Kubernetes internal network. I'm guessing fly.io doesn't just give you bare servers you can run k3s on, though. The only real difficulty here is hooking up totally different abstraction engines that aren't designed to work with each other. Putting the blame specifically on one of them and not the other doesn't make sense.
jauntywundrkind 2y ago
The pluggable storage (CSI) and networking (CNI) aspects of Kube tend to be a little limiting, in that a cluster generally only expects to have one at a time running. There is Multus, which is extremely well regarded CNI that enables mixing and matching different CNI Plugins. It works great, but it does involve diving into an additional level of complexity, and it can be exciting/unstable building the setup. Maybe the author didn't have rights over the cluster to do this or interest in mucking about deeply in this admittedly complex/highly capable system was limited. Or maybe just using an adding like this was appealing as non disruptive! This seems like a pretty creative & direct way to extend what they already had. There are good well tread options for higher power, for mixed modes of networking, but this strikes me as a nice way to add more without having to revamp the base cluster, which I found super cool & direct. https://github.com/k8snetworkplumbingwg/multus-cni https://github.com/k8snetworkplumbingwg/multus-cni

candiddevmike 2y ago

At its heart, Kubernetes is a workload scheduler, and a fairly opinionated one at that. IMO, it's too complicated for folks that just need an "extensible platform" when all they really need is a dumb scheduler API without all the Kubernetes bells and whistles that have been bolted on over the years. Kubernetes tries to be everything to everyone and makes the entire thing too complicated. Seeing Kubernetes broken up into smaller, more purpose built components and let folks pick and choose or swap could be helpful, at the risk of it becoming OpenStack. There's an alternate reality where Fleet is the defacto scheduler and cats and dogs live together in harmony.

mrgaro 2y ago
I'd say the opposite instead: we need Kubernetes distributions, just like Linux needs distributions. Nobody wants to build their kernel from scratch and to hand pick various user space programs. Same for Kubernetes: Distributions which pack everything you need in an opinionated way, so that it's easy to use. Now it's kinda build-your-own-kubernetes at every platform: kubeadm, EKS etc all require you to install various add-on components before you have a fully suitable cluster.
- p-o 2y ago
  I think the Operator pattern will grow to become exactly what you describe. We're still at the early stage of that, but I can see that a group of operators could become a "distribution", in your example.
- jen20 2y ago
  There have been distributions of Kubernetes for almost as long as there has been Kubernetes.
koito17 2y ago
> At its heart, Kubernetes is a workload scheduler Wouldn't it be more accurate to state Kubernetes at its core is a distributed control plane? You want something that inspects the state of a distributed system and mutates it as necessary to reach some "desired state". That's the control plane bit. For fault tolerance, the control plane itself needs to be distributed. This is the exact reason for Kubernetes' existence.

treflop 2y ago

What OP did could be done with any containerization platform that abstracted away networking, which should be all of them. Personally I think you should always be using containerization, because even single node Docker is easy. If you are running something for real, then definitely use Kubernetes. If you are using containerization, setting up Tailscale is trivial. Whenever you abstract away something, you can swap out core components like networking willy nilly. Of course, you can always over-abstract, but a healthy amount is wonderful for most non-very high performance use cases.