3 ms·
This makes me nostalgic of time we were on hadoop stack back in the day. Eco-system was thriving, hadoop, pig, hbase, hive, mahout. Everyday there was a new to
by debarshri 5y ago
This makes me nostalgic of time we were on hadoop stack back in the day.
Eco-system was thriving, hadoop, pig, hbase, hive, mahout. Everyday there was a new tool with an animal name released. Consultancies and VCs were bullish on the data tools. Even cloudera had a tagline that said - "Data is the new bacon". There were "BigData" experts in every organisation. Hortonworks was relatively new and was becoming popular. Somehow, all of them just disappeared in a span of 5 years. The market just shrunk or rather disappeared. Few survived - confluence, databricks and datastax.
Today, I see the same trend in kubernetes space. The eco-system is growing even faster than hadoop's eco-system. I have a feeling that at one point there will be realization that not everyone will need kubernetes and the market will start contracting and consolidating.
- Uberphallus 5y agoHi, working for major telco here. I think the realisation is that most data workloads aren't really big data, which people already knew back in 2015, but the same people will overengineer things just to learn stuff. The fad moved onto something else, I agree. But my department is a giant, just-ETL, everything-requires-a-cluster kind of department. More than half of our business runs on Hadoop, for a reason. The Hadoop ecosystem is a bit like K8s. If it feels too complicated for your purpose, it probably is. But with all its warts, when your data is big enough, either you use the Hadoop ecosystem or you reinvent significant parts of the wheel.
- debarshri 5y agoWhat tools and stack do you use in your organization?
- Uberphallus 5y agoFrom major technologies around Hadoop we use Hive, Oozie, Spark... it's easier to say the ones we don't use: HBase.