3 ms·
Our experience with Spark has been horrendous. Very unstable. Marginal improvements. Big hassle. Would strongly advise you to consider hadoop. We also used sto
by ampermad 12y ago
Our experience with Spark has been horrendous. Very unstable. Marginal improvements. Big hassle.
Would strongly advise you to consider hadoop. We also used storm and found it to be much stable.
Databricks makes a lot of noise though.
- kiyoto 12y agoCan you share the specifics of your experience (the cluster size/use case/etc.) I am partly asking this because you clearly feel strongly about this topic (your account was created an hour ago, most likely to comment on this).
- threeseed 12y agoHave to disagree on all points there. Spark on YARN has been very stable and zero hassle for us. Run one command and it is deployed around the cluster and up and running. For performance Spark absolutely destroys stock MapReduce/Tez especially if like us you have a cluster with lots of basically unused RAM. And Spark SQL, Spark Shell are both fantastic additions to the Hadoop ecosystem.