3 ms·

Why do you find this amusing?

• w_t_payne 13y ago
  I find it amusing that people are so sheep-like in the way that the herd mentality just takes over. To be fair, I am not claiming any sort of immunity from this effect, I am as guilty of it as the next person ... but it is quite funny in a sad way when you are able to step back and recognise it.
• mindcrime 13y ago
  It's basically what w_t_payne said. It's amusing (in a sense) because it betrays such a lack of understanding of what's actually going on, and such a willingness to simply accept the "received wisdom" and not question it and think independently.
  • dekhn 13y ago
    Some of us have backgrounds in MPI, BSP and other distributed computing paradigms. I switched to high throughput computing and I can tell you that I would never go back to MPI. People who use MPI almost always use it as a crutch to avoid solving the hard problem that would make their computation a lot easier. As an example I used to write MPI code for molecular dynamics simulators. The goal was to use a large supercomputer to scale the computation up to longer trajectories. But then, with cheap linux boxes it made more sense to just run many simulations in parallel, because low latency MPI class networks cost much more than the machines. We and others made the switch to other approaches which didn't need MPI- we ran many sims in parallel and did only loosely coupled exchanges of data. In fact the coupling was so loose we would run for weeks, dumping output in a large storage system and processing the data with MapReduce. When I look back at the people who still do the MPI simulations- they are doing dinky stuff and can't even process the data they generate because they spend all their time scaling a code using MPI that didn't need to be scaled with MPI.
    • mindcrime 13y ago
      Sure, and I'm not saying "MPI > Hadoop" or anything. Just pointing out that Hadoop is not the only game in town, and is hardly the only way to deal with "big data". I'm a Hadoop fan myself, but I also did a lot of MPI stuff in the past, and I believe there are still scenarios where MPI makes a lot of sense. I have less experience with OpenMP, but I think anybody planning a "big data" project would be well served to at least go out, and study up on all of Hadoop (and other M/R platforms), MPI, OpenMP, BSP, etc., before simply choosing Hadoop because it's the framework du jour.
      • gfodor 13y ago
        Often times choosing the framework du jour is the best choice just because it's the framework du jour. Support, training, books, an active ecosystem, a rich base of developers to hire from with experience, corporations incentivized to fund further development, etc, etc all act as a hedge against "slight technical mismatch between our requirements and the technology vs more esoteric ones."
        • mindcrime 13y ago
          But what you're talking about here is still a considered decision, based on actual analysis and thought. It's not choosing a framework only because it's the framework du jour, but because of the second order effects of it being so.