2 ms·
As some of the other comments have mentioned, kubernetes is “multitenant capable”, but the questions come down to: 1. Do you understand all the primitives and
by smarterclayton 7y ago
As some of the other comments have mentioned, kubernetes is “multitenant capable”, but the questions come down to:
1. Do you understand all the primitives and how they interact
2. Do your use cases require some or all of these primitives be bypassed (host network, root containers, custom extensions / operators), which may invalidate most security
3. Are the human costs to manage 1 and 2 less than the human/infra costs to manage multiple clusters?
Many applications have a few assumptions that violate “safety on a single machine” - databases use raw block devices, processes run as root by default. The harder it becomes to isolate an app on a single machine, the harder it is to isolate that workload on a cluster.
In addition, the moment you install something above kube (helm, ingress controllers, operators that can read or write secrets across namespaces) you start to blur the security boundaries at a cluster level and thus make it harder to reason about who can do what. It’s a lot like the Linux kernel / userspace separation - everything secure is in the kernel, but if a driver or subsystem has a vulnerability that tends to compromise the whole kernel.
I would suggest most people run multiple clusters until you get to the point where you have cluster sprawl (and are paying for a ton of redundant / idle infra), and then start either with an opinionated distro designed for picking all the defaults for you (OpenShift, a few of the other vendors) or hire someone who can design a top to bottom policy (and even then you’ll want multiple clusters for failure isolation).
- alexellisuk 7y agoI would agree with this. Soft multi-tenancy can work for some setups, until the tenant has a business need for something which could introduce a vulnerability and compromise nodes.
- drondin 7y agoI mostly agree with you that using current tooling seems like patching holes constantly but it's getting much better with sandboxing and automation is key and of course, it's not for everyone but can provide lots of benefits with "low" effort. There is a Kubernetes Working Group where the definition and implementation of soft multi-tenancy and any feedback is welcome.