4 ms·
I find a lot of the use cases end up using hadoop anyway and I was wondering why tools like oozie are not used. It appears as if such projects are feats of engi
by vikiomega9 11y ago
I find a lot of the use cases end up using hadoop anyway and I was wondering why tools like oozie are not used. It appears as if such projects are feats of engineering and nothing more. I might be gravely mistaken but that's how I see. Comments that suggest otherwise would be greatly appreciated. EDIT: I also find it odd that one might write a workflow manager because they can't find an equivalent one for python.
- suresk 11y agoCommon complaints I've heard about Oozie is that it has a high learning curve, not a great UI, and people hate the fact that it is XML based. This is a pretty decent comparison of Oozie vs Luigi (and Azkaban): http://www.slideshare.net/jcrobak/data-engineermeetup-201309 http://www.slideshare.net/jcrobak/data-engineermeetup-201309
- vikiomega9 11y agoThat presentation was pretty good with the good and bad takes. Do you think frameworks like Casacading or spark make things a lot easier as a higher abstraction on hadoop / different compute model?
- suresk 11y agoI haven't tried Cascading, but I've started doing some stuff with Spark and really like it. I feel like it is usually an easier abstraction to work with and it is a lot easier to prototype and experiment with.