4 ms·

I’m not sure who at Red Hat is advocating one cluster to rule them all, but it’s just one point on the spectrum. There are lots of places where one cluster mak

I’m not sure who at Red Hat is advocating one cluster to rule them all, but it’s just one point on the spectrum. There are lots of places where one cluster makes sense and two would be overkill - if you want to run lots of simple workloads, or have one very large scale app. But it’s equally smart to separate clusters by security domain or regulatory zone, or to create partitions to force your teams to treat clusters as fungible. If there’s Red Hat documentation advising silly absolutes please let me know and I’ll make sure it gets fixed.

jacques_chester 8y ago
I don't have an example to hand, so it's obvious I went on second-hand accounts. Do you have something you'd normally point customers to when describing the tradeoffs? For myself I see the argument for fewer clusters as about utilisation, the argument for more clusters about isolation. It's the oldest tug-of-war in computing. I think that shared node pools for multiple masters is going to be the combination that for most workloads will increase utilisation without greatly weakening isolation. I don't think multi-tenancy in the master will be as easily achieved or as effective.
- oso2k 8y ago
  In Red Hat OpenShift Consulting, we openly advise against “One cluster to rule them all” and the vast majority of our customers heed our advice. Our default delivery models support Sandbox, Nonprod, Prod cluster stand up. Some of us even support the idea that good IaC/EaC practices get our customers to where the cluster can be treated like cattle (much like pods and containers) in well-designed apps. My colleague Raffaele hinted as much when describing the problem as a matter of availability, disaster recovery and federation [0]. At least in OpenShift, multi-tenancy is a solved problem when cluster right-sizing has taken place. RBAC, node labels and selectors, EgressIP, quotas, requests and limits, multi-tenant or networkpolicy plug-ins go a long way. [0] https://blog.openshift.com/deploying-openshift-applications-multiple-datacenters/ https://blog.openshift.com/deploying-openshift-applications-...
  - jacques_chester 8y ago
    > In Red Hat OpenShift Consulting, we openly advise against “One cluster to rule them all” and the vast majority of our customers heed our advice. ... My colleague Raffaele hinted as much when describing the problem as a matter of availability, disaster recovery and federation [0]. To be honest, I should have realised this would be so. > At least in OpenShift, multi-tenancy is a solved problem when cluster right-sizing has taken place. RBAC, node labels and selectors, EgressIP, quotas, requests and limits, multi-tenant or networkpolicy plug-ins go a long way. Well, as you can guess, I am not convinced that this is really solved -- it looks like multiple discretionary access control mechanisms that need to be aligned properly, instead of a single mandatory access control mechanism to which other things align.
- ownagefool 8y ago
  It's also about tooling. I've seen many clusters being sold, but with no tooling to automatically build, monitor, secure and maintain these clusters, so you've got a DevOps team playing cluster wack-a-mole. Of course the consultancies love that because it's a bespoke layer for them to build and support, but the reality is setting up a small team to run a couple of clusters eases the job of discoverability and secops, and for many orgs is "good enough". Still, there is room for improvement, buy I doubt it's many masters without another product on top.
  - jacques_chester 8y ago
    > It's also about tooling. I've seen many clusters being sold, but with no tooling to automatically build, monitor, secure and maintain these clusters, so you've got a DevOps team playing cluster wack-a-mole. Pivotal's doctrine of how to use Kubernetes is explicitly multi-cluster oriented, but that's because we come to the table with tooling that excels at this kind of problem: BOSH.

jacques_chester 8y ago

I don't have an example to hand, so it's obvious I went on second-hand accounts. Do you have something you'd normally point customers to when describing the tradeoffs? For myself I see the argument for fewer clusters as about utilisation, the argument for more clusters about isolation. It's the oldest tug-of-war in computing. I think that shared node pools for multiple masters is going to be the combination that for most workloads will increase utilisation without greatly weakening isolation. I don't think multi-tenancy in the master will be as easily achieved or as effective.

oso2k 8y ago
In Red Hat OpenShift Consulting, we openly advise against “One cluster to rule them all” and the vast majority of our customers heed our advice. Our default delivery models support Sandbox, Nonprod, Prod cluster stand up. Some of us even support the idea that good IaC/EaC practices get our customers to where the cluster can be treated like cattle (much like pods and containers) in well-designed apps. My colleague Raffaele hinted as much when describing the problem as a matter of availability, disaster recovery and federation [0]. At least in OpenShift, multi-tenancy is a solved problem when cluster right-sizing has taken place. RBAC, node labels and selectors, EgressIP, quotas, requests and limits, multi-tenant or networkpolicy plug-ins go a long way. [0] https://blog.openshift.com/deploying-openshift-applications-multiple-datacenters/ https://blog.openshift.com/deploying-openshift-applications-...
- jacques_chester 8y ago
  > In Red Hat OpenShift Consulting, we openly advise against “One cluster to rule them all” and the vast majority of our customers heed our advice. ... My colleague Raffaele hinted as much when describing the problem as a matter of availability, disaster recovery and federation [0]. To be honest, I should have realised this would be so. > At least in OpenShift, multi-tenancy is a solved problem when cluster right-sizing has taken place. RBAC, node labels and selectors, EgressIP, quotas, requests and limits, multi-tenant or networkpolicy plug-ins go a long way. Well, as you can guess, I am not convinced that this is really solved -- it looks like multiple discretionary access control mechanisms that need to be aligned properly, instead of a single mandatory access control mechanism to which other things align.
ownagefool 8y ago
It's also about tooling. I've seen many clusters being sold, but with no tooling to automatically build, monitor, secure and maintain these clusters, so you've got a DevOps team playing cluster wack-a-mole. Of course the consultancies love that because it's a bespoke layer for them to build and support, but the reality is setting up a small team to run a couple of clusters eases the job of discoverability and secops, and for many orgs is "good enough". Still, there is room for improvement, buy I doubt it's many masters without another product on top.
- jacques_chester 8y ago
  > It's also about tooling. I've seen many clusters being sold, but with no tooling to automatically build, monitor, secure and maintain these clusters, so you've got a DevOps team playing cluster wack-a-mole. Pivotal's doctrine of how to use Kubernetes is explicitly multi-cluster oriented, but that's because we come to the table with tooling that excels at this kind of problem: BOSH.