3 ms·
That article totally misrepresents the normal use case for a Hadoop cluster though. Hadoop clusters are meant for when you have multiple petabytes of data and t
by saberience 8y ago
That article totally misrepresents the normal use case for a Hadoop cluster though. Hadoop clusters are meant for when you have multiple petabytes of data and this network bandwidth becomes the bottleneck for doing batch processing jobs.
Let me know when your command line tools run multiple large scale processing jobs on petabyte datasets.
His article was him constructing a straw man about why people use Hadoop and attacking that.
- henryfjordan 8y agoI've worked with these "straw men" you say he constructed, they are absolutely out there. There was a time in 2014 when Hadoop/MapReduce was the hammer and every problem out there looked like a nail. How many people have used Hadoop for a project? How many Petabyte+ datasets do you think are out there? Unless you truly believe the answer to those 2 questions are the same, I think you can see why that article had to be written.
- lloydde 8y agoI feel it is in a similar spirit as the OP here. In both articles the important take away for me is focusing on an approach and tools that match the problems. It sounds like you are making a similar point. I also appreciate quality of the writing and step by step journey. When I first read the article, I found it timely as I felt bombarded by IaaS & PaaS offerings with “web scale” solutions for problems I couldn’t relate to, basic examples, and few case studies.