Oh man, my org at FB was the test bench for moving to containerization and fancy service discovery and all that (Tupperware, just a different Borg reimplementat

Oh man, my org at FB was the test bench for moving to containerization and fancy service discovery and all that (Tupperware, just a different Borg reimplementation). It eventually got pretty good, but it never stopped being mad overkill for single-thousands of boxes. When you’re in the 10s or 100s of thousands in a fleet, or when you’ve got workloads that don’t neatly slot into your SKUs, Kube/Borg/TW/Mesos are The Way. No doubt about it. But it’s always seemed zany to me to stack namespaces/cgroups/etc. on top of Xen or whatever EC2 is using. Yo dawg I heard you like an abstract machine so I put… There are just separate concerns: - reproducibility (shared libraries argggghh) - resource limits to bin-pack SKUs - service discovery - failover - operational affordance And I’ve seen it get so ugly to conflate these very different imperatives. Running 100Ks that need to web serve on demand but web index when idle? Yeah, now you need the whole enchilada. But it’s false and harmful to promote the idea that the minute you need Grafana or DNS or Salt/Ansible/Nix/whatever that you need BORG. There are scenarios where I would enthusiastically break out Kube, but most of the marketing around it falls into my “and you will do nothing, because you can do nothing.” bucket.