2 ms·
Sure, and I'm not saying "MPI > Hadoop" or anything. Just pointing out that Hadoop is not the only game in town, and is hardly the only way to deal with "big d
by mindcrime 13y ago
Sure, and I'm not saying "MPI > Hadoop" or anything. Just pointing out that Hadoop is not the only game in town, and is hardly the only way to deal with "big data". I'm a Hadoop fan myself, but I also did a lot of MPI stuff in the past, and I believe there are still scenarios where MPI makes a lot of sense. I have less experience with OpenMP, but I think anybody planning a "big data" project would be well served to at least go out, and study up on all of Hadoop (and other M/R platforms), MPI, OpenMP, BSP, etc., before simply choosing Hadoop because it's the framework du jour.
- gfodor 13y agoOften times choosing the framework du jour is the best choice just because it's the framework du jour. Support, training, books, an active ecosystem, a rich base of developers to hire from with experience, corporations incentivized to fund further development, etc, etc all act as a hedge against "slight technical mismatch between our requirements and the technology vs more esoteric ones."
- mindcrime 13y agoBut what you're talking about here is still a considered decision, based on actual analysis and thought. It's not choosing a framework only because it's the framework du jour, but because of the second order effects of it being so.