I use managed k8s, but have sympathy for the bare metal folks. Wiring servers together using kubeadm is so simple and powerful that the missing pieces of the pu

I use managed k8s, but have sympathy for the bare metal folks. Wiring servers together using kubeadm is so simple and powerful that the missing pieces of the puzzle (networking and storage, for example) feel frustratingly close. In an ideal world, you could purchase or rent commoditized resources (compute, storage, bandwidth) and have those resources deploy your application in a standardized way. K8s is so close to that out of the gate, that having to use cloud provider's expensive and proprietary offerings feels sub-optimal. I agree that cloud offerings are the way to go for anyone without good reasons not to (eg compliance, lack of connectivity, or full control over upgrades). But a lot of the "I prefer on prem" attitude comes from realizing that having a black box of an ingress controller on my cluster isn't the end state of how apps get deployed