4 ms·
Insert "almost" into parent's sentence and it becomes correct. Many people use/used Hadoop for problems that did not warrant the overhead and complexity that c
by stingraycharles 7y ago
Insert "almost" into parent's sentence and it becomes correct.
Many people use/used Hadoop for problems that did not warrant the overhead and complexity that comes with Hadoop. I've seen it countless times with my own eyes that people pre-emptively use tools like Hadoop and Spark because of a chance that they will hit a massive scale in the future.
This happens in both startups and enterprises alike: people like to think they have big problems too often.
- jabl 7y ago> This happens in both startups and enterprises alike: people like to think they have big problems too often. A.k.a. resume-driven development. Having Hadoop on your CV looks sexier than awk.
- arethuza 7y agoI was at a conference once where someone was presenting and describing something where they used Hadoop to do pretty simple processing on ~30MB of data and the audience appeared to be lapping it up. I just sat there thinking I could probably run what they did on my phone. And just to be clear - it wasn't a PoC or a demo.
- coredog64 7y agoThis. A prior employer spent millions on a Cloudera install that had 5 worker nodes and was being used to service a bunch of generic jobs. Worse still, it didn’t even use HDFS and we eventually got sick of the crappy embedded Zookeeper/Kafka setup.
- geezerjay 7y ago> Insert "almost" into parent's sentence and it becomes correct. No, it still remains astonishingly wrong. Even container orchestration platforms are being adapted to provide the same service that Hadoop has been providing for years, and no one in their right mind would claim that running processing jobs on the cloud is a problem that almost no one has.