Why would Hadoop automatically be the right tool for the job in this case? In fact, when is any tool automatically the right tool for the job? I understand th

Why would Hadoop automatically be the right tool for the job in this case? In fact, when is any tool automatically the right tool for the job? I understand the desire not to do unnecessary work and to plan ahead. However, something like Hadoop is very heavy weight and requires a significant change to the way you write your software. It isn't something easy to undo if Hadoop turns out to be the wrong direction. Thus, if you start there you are more likely to stay there, even if "there" is not appropriate. On the other hand, starting with a much simpler system and adapting it as your needs change may add a little more work over the course of the project, but saves a buttload of necessary technical debt.