4 ms·
I sort of agree. However when you think you will grow into the TB range, you just as well do it in hadoop right away. We are using Hive and HiveQL and have SQL
by TeeWEE 13y ago
I sort of agree. However when you think you will grow into the TB range, you just as well do it in hadoop right away.
We are using Hive and HiveQL and have SQL like queries which generate the correct output. The result is: we dont have to hassle with the hadoop mappers and reducers. And we can write our "queries" in a human readable fashion.
- vonmoltke 13y agoWhy would Hadoop automatically be the right tool for the job in this case? In fact, when is any tool automatically the right tool for the job? I understand the desire not to do unnecessary work and to plan ahead. However, something like Hadoop is very heavy weight and requires a significant change to the way you write your software. It isn't something easy to undo if Hadoop turns out to be the wrong direction. Thus, if you start there you are more likely to stay there, even if "there" is not appropriate. On the other hand, starting with a much simpler system and adapting it as your needs change may add a little more work over the course of the project, but saves a buttload of necessary technical debt.
- TeeWEE 13y agoI do agree with you. I prefer too keep it simple too, and not plan ahead too much According to the KISS principle. I'm not saying hadoop is the solution all the time. I'm just saying that sometimes it makes sense.
- TeeWEE 13y agoI do agree with you. I prefer too keep it simple too, and not plan ahead too much According to the KISS principle. I'm not saying hadoop is the solution all the time. I'm just saying that sometimes it makes sense.