Disclaimer : I work for a corporate IT consulting giant. The trend I'm seeing the big Data sphere in my company is a by-and-large move away from technologies t

Disclaimer : I work for a corporate IT consulting giant. The trend I'm seeing the big Data sphere in my company is a by-and-large move away from technologies that implement MapReduce and and complicated batch processing with HDFS as a data store. More and more customers want insight as soon as they get/produce their data so we've seen a particularly large increase in interest in technologies such as Kafka/ Samza and spark/ pySpark. I see a trend in Kafka and but I think the community needs to jump behind it too, keep it as a pipeline tool and not a querying engine. I dont see Hadoop-based solutions going away any time soon though.