4 ms·

Thanks Andy! To be clear, I really appreciate your effort to create a better platform for big data. I have spent 10 years on trying to make Hadoop & Spark finan

Thanks Andy! To be clear, I really appreciate your effort to create a better platform for big data. I have spent 10 years on trying to make Hadoop & Spark financially and technically scalable for companies with more than 1PB data and I totally agree with you that we need a better system. I am just not ready to trade Hadoop or Spark problems to Kubernetes problems. The question is what orchestration do we need? What model do you want to implement? Could we build a better Kubernetes?

andygrove 7y ago
I've only been using Kubernetes for a couple months so far and am still learning, but I am very impressed so far. I love the way it facilitates dev and devops collaborating and the fact that it is cloud agnostic (I can even run a Kubernetes cluster on my desktop for local testing). My opinion so far is that the distributed cluster part is really solved by Kubernetes.
- StreamBright 7y ago
  Yeah it is absolutely amazing for exactly that and yes it has a good approach to the distributed cluster problem. Once you put it in production there a very different picture. https://github.com/hjacobs/kubernetes-failure-stories https://github.com/hjacobs/kubernetes-failure-stories I haven't had a single Kubernetes user who had it in production and did not have stability or performance issues with it. This is why I mentioned that I am not ready to trade my Hadoop outages and performance issues for Kubernetes ones.