> We manage LVM ourselves, yes. We don't do any live migration. Our disk offering is relatively new so we haven't exposed much snapshot based functionality to t

> We manage LVM ourselves, yes. We don't do any live migration. Our disk offering is relatively new so we haven't exposed much snapshot based functionality to them. Most of them just want to know backups are happening and disks are encrypted, though. Reasonable -- if all the workloads run in containers to start with and are service-y probably not a real need, and certainly not a real need if no customer has asked for it/had it be a deal breaker.. > Network isolation was much simpler with a little bit of bpf and a ipv6 tricks. Some of that is detailed here: https://fly.io/blog/ipv6-wireguard-peering/ https://fly.io/blog/ipv6-wireguard-peering/ > we'd have been fighting cni complexity to make it work. Appreciate the candid responses, thanks for taking the time. That ipv6 wireguard peering post was really fascinating I read that too. Wireguard has been quite the a game-changer in it's space as well and a lot of value IMO is just in the simplicity and difficulty of misconfiguration, even though the performance is also fantastic. Grateful that ya'll are sharing what you're doing right/finding interesting. Since ya'll might appreciate this, I think there's an ultimate form of all these orchestrators out there that boils everything down to the "operator pattern" -- I call it "buhzaar" but I tried to get my thoughts out of the notebook a while ago[0]. It's almost like a completely normalized DB might be -- to strip an orchestrator down to it's bare minimum, which facilitates other processes that do resource provisioning and management. Then let people bring their own things that provision resources (and maybe you some "officially supported" ones but they all live separately and iterate separately). I didn't quite put down all the thoughts I had but you think this is too much normalization (in the same way no one wants to do 7 joins)? You could argue that both nomad and k8s are denormalized (they intrinsically "know" how to provision/manage certain things) to a certain extent, and nomad just "bundles" less. [0]: https://gitlab.com/buhzaar/design https://gitlab.com/buhzaar/design