6 ms·
Vcluster – Create fully functional virtual Kubernetes clusters
- linkdd 5y agoAwesome! Seems neat to spin up temporary lightweight clusters for E2E tests in a CI/CD pipeline, instead of a KinD cluster which can have very low performances.
- gentele 5y agoEphemeral clusters for dev and CI/CD is the main use case right now but due to the fact that the pods don't take a performance hit - as you already mentioned - because they are scheduled in the underlying cluster, vclusters can also be a very interesting concept for production workloads, effectively sharding your k8s cluster into logical clusters / control planes.
- nhoughto 5y agothe only problem with this is needing to push your container images in CI before being able to schedule, as the real pod will a real node, which will need to pull a container. In kind you can build container images into the single host, and avoid a container push/pull cycle. Any solns to that in vcluster?
- GauntletWizard 5y agoEverything old is new again; I remember when Borgproxy allowed you to schedule jobs to "whichever cluster had the least load" within a region, and I imagine this is implemented and works similarly.
- gentele 5y agovcluster maintainer here. Right now, it is more about multiple vcluster running on the same Kubernetes cluster which addresses the multi-tenancy issues of Kubernetes. However, a single vcluster with multiple Kubernetes clusters to schedule workloads on is a super interesting idea as well. Federated clusters was a big topic a few years ago but it is very hard to solve.
- cbsmith 5y agoAll problems in computer science can be solved with another level of indirection.
- jrockway 5y agoNot really on-topic but does anyone have a "fake" Kubernetes? More than just an API server that serves canned responses, but something that actually runs the control loops but doesn't run any containers. For example, I could configure it in advance like: containers: image: foo/bar:1.2.3 produce_logs: "hello this is fake foo/bar" Then kubectl apply a manifest like: apiVersion: apps/v1 kind: Deployment metadata: name: foobar spec: replicas: 1 template: spec: containers: - name: foobar image: foo/bar:1.2.3 Then run a command like "kubectl get pods": NAME READY STATUS RESTARTS AGE foobar-abcd123456-f0ob4 2/2 Running 0 1s Or even "kubectl get logs foobar-abcd123456-f0ob4": hello this is fake foo/bar! Basically, this would let you test things that interact with kubernetes, without having to have a cluster. This thing would just be a library that you link against your tests, or a program that you start up next to your tests, and it gets you most of the way there. It's better than a fake apiserver you write yourself, where you "know" that the service controller looks at pods that match a selector, then produces endpoints objects that contain that IP address; you could write this kind of low-overhead test without having to know those things (or get it wrong, and code your app to a k8s spec that doesn't exist except in your own mind). That's what I'd really like.
- whacker 5y agoNot quite fake: but this might be close enough? https://github.com/kubernetes-sigs/kind https://github.com/kubernetes-sigs/kind Alternatively you could create a CRI plugin that does what you want. https://github.com/kubernetes/kubernetes/blob/242a97307b34076d5d8f5bbeb154fa4d97c9ef1d/docs/devel/container-runtime-interface.md https://github.com/kubernetes/kubernetes/blob/242a97307b3407...
- jrockway 5y agoYeah, kind is a good compromise. It's more than I want and it does take some time to startup, but it's pretty lightweight and very accurate. Because everything is running locally you can use tools like `perf` to keep an eye on things you're running on k8s, right from your usual workstation. I like it a lot. The CRI plugin sounds like probably the right thing to do. Everything is real up until actually running stuff.
- rektide 5y agoOh my god, I'm so happy I could cry. > Each vcluster runs inside a namespace of the underlying k8s cluster. . . It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces. Kubernetes has invented a bunch of their own means of measuring & consuming resources, and everything under Kubernetes all runs inside one single namespace, hopefully carefully managed (by human operators who carefully make sure things are going ok, or else). I keep feeling like I'm missing something, that I have to be wrong, because Linux has really good ways to divide workloads. Namespaces. You put different jobs or different classes of work into different namespaces. Weirdly Kubernetes is most unhelpful about doing this job, about assisting with this subdivision process. This is the first project I've seen in Kubernetes land that explicitly names the clear, obvious, wonderful step of using multiple Linux namespaces to divide up work, as a way to more reasonably achieve multi-tenancy & separation of workloads. The first Kubernetes project I've heard that lets the Linux kernel do what it's good at. I'm not sure what the use is for corporations or enterprises. I think dividing up resources amid different teams is a reasonable use. But I am incredibly excited for hyper-converged home-labs & other over-growing diy computer-science projects to start having a good way to host a variety of tenants in a reasonably controlled, semi-standard way. Introducing multiple namespaces is something Kubernetes has desperately needed. I am head over heels in love with what's happening here. I hope the implementation doesn't get too much wrong, because Kubernetes has ignored the best way to divide resources up on a computer, has done too much on it's own, within one namespace, and starting to expand across multiple namespaces is a game changer, opens the field for how useful and how interesting Kubernetes is.