I've ran several small postgres databases and a kafka cluster on Kubernetes; in our case, the VM route would have added lots and lots of overhead with very ques

I've ran several small postgres databases and a kafka cluster on Kubernetes; in our case, the VM route would have added lots and lots of overhead with very questionable benefits, given that we didn't run a single prod VM ourselves (all managed by GKE) and hence we had good tooling, observability, monitoring for k8s workloads in place, but VMs were essentially treated as more like a set of disposable blackboxes (not entirely, but we'd only monitor certain aspects and leave all else to GKE) Being able to get essentially the same result, but with our familiar processes, tooling, infra, that was really helpful getting things off the ground quickly and keeping them stable. We picked k8s/GKE way before that as a platform where a small set of abstractions allow us to do a lot of different things, and that makes it easy to debug unfamiliar workloads, keeping operations overhead low, and it worked out really well, and much easier for people with little ops experience to learn and get to a point where they could take care of their workloads themselves. And it never felt we had to mangle k8s for our stateful workloads at all. Statefulsets, storage, node affinity etc. feel very much like organic building blocks and work really really well, with very few surprises. I guess k8s has come a long way as far as stateful workloads are concerned.