Kubernetes has worked really well for us, it's given use distributed systems primitives that just work which has allowed us to focus on the parts of the system

Kubernetes has worked really well for us, it's given use distributed systems primitives that just work which has allowed us to focus on the parts of the system that are unique to what we're doing. We get a lot of questions about why we choose to implement our own filesystem and computation layer and the answer we gave is the one you have here, it allows us to make sure they work well together because they're built from the ground up with containerized workloads in mind. We could have tried to containerize Hadoop, but I doubt it would have resulted in the fully integrated system that we have today. Too much in the Hadoop ecosystem was written for a precontainer world for it to make sense to containerize it after the fact.