36 ms·
About 10 years ago, I worked at a place where we had a "big data" project. Some big wig wanted to use Hadoop. Turns out, the "big data" isn't big. It's not even
by icedchai 3y ago
About 10 years ago, I worked at a place where we had a "big data" project. Some big wig wanted to use Hadoop. Turns out, the "big data" isn't big. It's not even a gigabyte, spread over less than a dozen files. Everything worked great, but the job orchestration took longer than the actual processing.
Despite this misuse of Hadoop, another guy really loves it, and decides to start a project rewriting everything to use MapReduce. A new guy started, got assigned to the "MapReduce project"... he worked on this for over a year. It never made it to production.