3 ms·
Curious about what your concerns with Spark were? I work for a company that supports Spark development, but I don't work closely with that project, so my opinio
by TallGuyShort 11y ago
Curious about what your concerns with Spark were? I work for a company that supports Spark development, but I don't work closely with that project, so my opinion is not sufficiently well-informed, and obviously biased.
As far as I know, virtually any MapReduce job can be rather trivially translated to Spark's .map() and .reduce() operations. And the downsides are: it's model hasn't yet been proven at the largest scale's MapReduce has been used, and possibly the use of Scala (although Java / Python bindings are obviously available). Were there any other major factors in your hesitance?
- mark_l_watson 11y agoI didn't have concerns about Spark, rather I already felt comfortable with Hadoop. Another issue is that I am sort of retired now. I still accept small consulting jobs and do a lot of writing but my technology choices have shifted to fun things like Pharo Smalltalk, Haskell, etc.
- boothead 11y agoIf you're interested in this domain and Haskell and have some free time, you might find hailstorm interesting [1] From what I understand of it - it's an implementation of apache storm in Haskell. [1] http://hailstorm-hs.github.io/hailstorm/demo/#1 http://hailstorm-hs.github.io/hailstorm/demo/#1
- rshaban 11y agoNeat! Thanks for sharing this, it's a great overview.