4 ms·
I'm not really familiar with this industry. Are these Hadoop clusters hosted locally by these businesses? And MapR et all were providing the software/consulting
by dmix 7y ago
I'm not really familiar with this industry. Are these Hadoop clusters hosted locally by these businesses? And MapR et all were providing the software/consulting to help manage them?
Then I'm guessing better cloud hosted options came out offering similar capabilities.
If that's the case was it really a big surprise that "cloud" hosting would eat any self-hosted platform's lunch?
- capkutay 7y agoyes mostly locally hosted back then. the idea was you'd have this massive distributed file system to hold all your data, and VC backed comopanies like Cloudera, MapR, Hortonworks promised everyone that they'd build the SQL layer, the data warehousing, BI, and all the other enterprise features you'd need to basically replace your expensive Teradata, Oracle Exadata, and other data warehousing systems. But of course it didn't play out that way.
- threeseed 7y agoActually it did play out that way. Most enterprises these days will have a massive distributed file system (S3 or equivalent) that has most of their data. And most will be running a decent percentage of data transformation jobs using Spark or some SQL layer e.g. Presto and then running BI tools like Tableau using Athena/Redshift Spectrum (or equivalent) as their SQL layer. It's just that this is all playing out on the cloud instead of on-premise with some vendor. But you have definitely been seeing the decline of the core enterprise data warehouse.
- threeseed 7y agoSo in the very recent past (i.e. a few years) businesses wanting to do Data Science ran a distro of Hadoop e.g. MapR that they bought from the vendor. They charged an exorbitant charge per node (e.g. $10k) because they figured they would get people to switch from Teradata or Oracle. Now what the cloud offered was so much more compelling. You had per hour pricing on the order of $20 for a minimal cluster. You had unlimited autoscaling so you didn't have to do capacity planning and go through procurement processes to pre-order hardware/licenses. And of course you had unlimited, ultra-cheap storage courtesy of S3. And it also allowed each team to have their own mini-cluster instead of everyone relying on some giant one. I don't think the recent explosion in Data Science would've happened without the cloud.
- disgruntledphd2 7y agoWhich is hilarious, given how expensive it is to run analytics on AWS. My current company looked into recreating our internal analytics environment on the cloud, and realised that it wouldn't be cost effective.
- notyourday 7y agoYes, MapR was local hosting.