4 ms·

I'll go with separate Kubernetes clusters. The cost is more systems to manage, but they will be simpler and pretty much identical, and some low but not zero res

by zaat 7y ago

I'll go with separate Kubernetes clusters. The cost is more systems to manage, but they will be simpler and pretty much identical, and some low but not zero resource waste with each cluster you add. Ill add that there are few well established companies that have adopted this approach, AWS, Google and Azure to name a few.

polskibus 7y ago
In this approach, is a single VM node reused by multiple kubernetes clusters or is a node exclusive to a single cluster? If only the latter is possible then what is the best way to maximize resource utilization in cluster-per-tenant scenario? What happens if new customer wants his tenant - is automating of provisioning of an entire new cluster feasible?
- zaat 7y ago
  Each vm belongs to a single cluster. The best way to maximimize resource utilization is to size the vms according to the tenant load, generally you want to minimize the number of vms, as each of them add some overhead, but you don't want to waste unnecessary resources. You must have a minimum set of vms in each cluster to ensure high availability. Automating the provisioning of an entire new cluster is very easy, at least for some people with the required resources. But even for a small shop it is possible to automate it using stuff like Jenkins and Ansible. It is still relatively sophisticated stuff. It is not rocket science but not an elementary school project level either.
  - polskibus 7y ago
    Thanks, that was informative. Do you have a reference that you'd recommend about the details of how the big names that you've mentioned are doing multitenant Kubernetes? I'd love to read more about it.
    - zaat 7y ago
      Oh, sorry, that was tongue in the cheek. They are cloud providers, and as such they provide costumers with separate cluster. The aks-engine repo let you peep to what goes behind the scenes, but this is not the kind of beast you will grow at home - it is a public cloud provider with global scale, with the expected level of complexity and engineering. It is available on: https://github.com/Azure/aks-engine https://github.com/Azure/aks-engine
    - merb 7y ago
      btw. in hard multitenant k8s you can even go as wide as using a "master" cluster that contains the control plane of another cluster. actually you need the right network cni for that.
- jacques_chester 7y ago
  > What happens if new customer wants his tenant - is automating of provisioning of an entire new cluster feasible? This is doable through various provider/vendor-specific APIs. Pivotal's PKS has an API. GKE has one, EKS, AKS and so on have them too. In the longer run there's meant to be a full Cluster API, which I presume will take over for that role: https://github.com/kubernetes-sigs/cluster-api https://github.com/kubernetes-sigs/cluster-api

by zaat 7y ago

polskibus 7y ago

In this approach, is a single VM node reused by multiple kubernetes clusters or is a node exclusive to a single cluster? If only the latter is possible then what is the best way to maximize resource utilization in cluster-per-tenant scenario? What happens if new customer wants his tenant - is automating of provisioning of an entire new cluster feasible?

zaat 7y ago
Each vm belongs to a single cluster. The best way to maximimize resource utilization is to size the vms according to the tenant load, generally you want to minimize the number of vms, as each of them add some overhead, but you don't want to waste unnecessary resources. You must have a minimum set of vms in each cluster to ensure high availability. Automating the provisioning of an entire new cluster is very easy, at least for some people with the required resources. But even for a small shop it is possible to automate it using stuff like Jenkins and Ansible. It is still relatively sophisticated stuff. It is not rocket science but not an elementary school project level either.
- polskibus 7y ago
  Thanks, that was informative. Do you have a reference that you'd recommend about the details of how the big names that you've mentioned are doing multitenant Kubernetes? I'd love to read more about it.
  - zaat 7y ago
    Oh, sorry, that was tongue in the cheek. They are cloud providers, and as such they provide costumers with separate cluster. The aks-engine repo let you peep to what goes behind the scenes, but this is not the kind of beast you will grow at home - it is a public cloud provider with global scale, with the expected level of complexity and engineering. It is available on: https://github.com/Azure/aks-engine https://github.com/Azure/aks-engine
  - merb 7y ago
    btw. in hard multitenant k8s you can even go as wide as using a "master" cluster that contains the control plane of another cluster. actually you need the right network cni for that.
jacques_chester 7y ago
> What happens if new customer wants his tenant - is automating of provisioning of an entire new cluster feasible? This is doable through various provider/vendor-specific APIs. Pivotal's PKS has an API. GKE has one, EKS, AKS and so on have them too. In the longer run there's meant to be a full Cluster API, which I presume will take over for that role: https://github.com/kubernetes-sigs/cluster-api https://github.com/kubernetes-sigs/cluster-api