16 ms·
Cloudera and Hortonworks merge
- ameyamk 8y agoGood move! Would help consolidate Hadoop ecosystem. MapR would find it hard to compete now. Both these companies are being challenged with a new crop of Databricks and the confluents of the world.
- nerpderp83 8y agoYikes!
- medh2000 8y agoI have both used Cloudera stack and Hortonworks. I would say in my experience that installation and management is much straightforward with HW distribution. Both companies stacks have similar Apache projects and management software like Ambari and Cloudera Manager.
- nerpderp83 8y agoIt doesn't bode well for either of them. This is a pure contraction of the markets they are trying to serve.
- xab9 8y agoThat's interesting. Been at their Hungarian office for a job interview (Hortonworks, a year or so ago), it was weird though. Haven't tried Cloudera, but they too have an office at Budapest, I wonder how it will effect their workforce in the region.
- SimplyUseless 8y agoThis is not to eliminate threats from new comers but to find an identity in the new era of cloud.
- Thaxll 8y agoHadoop very "fancy" 5-6 years ago, what's the trend now? I guess with managed services from AWS / Google it makes Hadoop less useful?
- deleted 8y ago[deleted]
- threeseed 8y agoThe managed data science services from AWS/Google is Hadoop/Spark.
- medh2000 8y agoTrue. Some one in the cloud is managing these Hadoop stacks for you. You just pay usage of these services.
- tumanian 8y agoIts still there - storage is still mainly HDFS, but computation layer morphed into Spark on Yarn (Mesos as a scheduler is barely used). Barely anyone runs classic Mapreduce jobs, Spark all around, written in Scala(or java or python).
- francoisprunier 8y agoInterestingly, none of them mention Hadoop. It makes sense to merge from a business point of view, and on a technical level it could finally mean more work on polishing the gazillion tools they include, which is badly needed.
- capkutay 8y agoGood move for both companies. The surplus of 'enterprise hadoop' companies was created by a mixture of hype and a peak in VC investment in open source. The fact that Cloudera, Hortonworks, MapR were all founded and raised $100m+ around the same time was a bit superfluous for the whole market. Hard to say where this leaves MapR now. They seem to be the odd man out in terms of growth and adoption.
- jamesblonde 8y agoWe have a startup, Logical Clocks (www.logicalclocks.com), that just raised money to sell our data-science next-generation Hadoop stack. It has a new verison of HDFS called HopsFS (Nvme storage, distributed metadata) and support for GPUs in YARN. And distributed tensorflow, Spark, Airflow, Flink. So, what does that mean for us? I am shellshocked. I expect prices to increase (good for us). What else should we expect?
- agibsonccc 8y agoBoth Horton and Cloudera require a huge number of partner resellers and consultants. What horton and cloudera have isn't just software, but brand, an install base, and the ability to back it up. They merged partially because they were both being cannibalized by services revenue they couldn't get rid of. Now they are struggling to move to the cloud (see Atlas now) I'm not sure yet another hadoop distro with a bunch of 1 off tooling that is supposedly faster is the answer. Why is all that stuff even needed?
- jamesblonde 8y agoAdam, what about Java for deep learning? Is that needed?
- agibsonccc 8y agoAccording to my customers and users yes? Granted, we do a lot more than just that though. Dl4j itself has a decent sized user base. Ranging likely from your phone maker to your bank and retail store. We have our own software distro too which is why I'm commenting on this. We don't try to boil the ocean with a bunch of tech though. There's a whole new crop of companies focusing on solving bits of the ML problem well rather than trying to do storage and god knows what else. My point here about you guys is you're trying to compete in what is largely a commodity market. People don't need all this stuff. Simplicity won here. It's not about better tech. You guys have the same pitch MapR does and largely the same problem: Better tech is only part of the problem with adoption. You need customers, users, and a clear business model when going to market. Cloudera and Horton ran one playbook that at least somewhat worked (it got them public) and now they can focus on competing with the cloud vendors, which made the right decision and just made commonly used software easy to use.
- alexnewman 8y agoI was an early employee at Cloudera, am a Hadoop contributor and think this entire market is garbage. Basically the big data field went off the rails and this is Cloudera's way of trying to remain relevant. It's hard to point to a product that came after my tenure (I was on the team that original made the POS called Cloudera Manager) that's really used by anyone at scale. Cloud is displacing all of these tools and never got the clouds to play ball.
- colin_mccabe 8y ago> It's hard to point to a product that came after my tenure (I was on the team that original made the POS called Cloudera Manager) that's really used by anyone at scale. There are tons of them. Spark, Hue, Sentry, Kafka, the list goes on.
- dswalter 8y agoI think the parent comment was saying that none of these were primarily made by cloudera.
- alexnewman 8y agoindeed. or controlled them
- colin_mccabe 8y agoHue is a Cloudera project, not even an Apache project. Sentry is an Apache project but primarily developed by Cloudera.
- alexnewman 8y agoHue was made before cloudera manager. I know because when I first wrote it, it was based on hue. I think if sentry and hue are all cloudera has brought to the table...
- alexnewman 8y ago
- Tsarbomb 8y agoCan someone explain to me what the big draw was for Hortonworks or Cloudera? Working as a lead in a small team that deals with a colossal amount of data (human genomics), it was always easier for us to hand roll deployments with terraform/ansible in either baremetal or OpenStack environments. In the public clouds like AWS we are using the managed services like EMR. The whole sales pitch I've always got from either hortonworks or Cloudera always seemed more aimed at nontechnical stakeholders than the technical ones. Am I wrong? Have I missed out on some cool stuff?
- sfohohoho 8y agoIt's cool bro. You obviously are a np. Million dollar Cloudera deployments with 1000s of ba users. You missed out on working for a large, for profit, business. Not a big deal.
- ewencp 8y agoterraform: 2014 ansible: 2012 openstack: 2010 cloudera: 2008 hortonworks: 2011 emr: 2011 (those were quickly googled, apologies if i got any dates wrong) timing is important. cloudera had good, early timing and looked promising because of it. you are right that EMR definitely hurt all the other hadoop vendors, though i think people overestimate how comfortable big enterprises are with moving to public cloud. way more comfortable today; back then everything was a lot less certain. cloudera's name is unfortunate given they never got anything cloud-based successful, maybe they'd be in a better place now if they had. but some of the key technologies you suggest as better options came 4-6 years later. that's 4-6 years of providing value, gaining traction, and building a committed customer base. 4-6 years is a long time, and even with how slow many enterprise projects run, more than enough to get entrenched, build tooling that makes a bunch of stuff easier, build mindshare, etc. > Have I missed out on some cool stuff? stuff that makes companies money doesn't always look cool.
- rawoke083600 8y agoPlain-Old-Bash-Scripts 2018 ~
- coredog64 8y ago
- thinkmassive 8y ago> The two companies are committed to supporting existing offerings from the two companies for at least three years but will work on a "unity release" of software, drawing on technologies from both companies' portfolios, Reilly said. I’m really intrigued to see whether Ambari or Cloudera Manager wins out in the long run. This is unexpected but interesting. (I worked at Hortonworks 2014-2016, no current affiliation)
- medh2000 8y agoThat's a good question. Both CM and Ambari are front end for managing all the services and more.
- jamesblonde 8y agoThis is, of course, an anti open-source move. So, Cloudera Manager will win, there. The stack will go largely proprietary, as this is a defensive move - AWS are just killing them in the cloud. They need to prevent AWS just repackaging the Hadoop core and reselling it - what better way than by making your new platform AGPL-v3 or something, so you're still open-source, but AWS can't just resell it as EMR.
- sfohohoho 8y agoNo one saw this coming... for years Intel just held it up for a bit more profit.
- docker_up 8y agoLots of overlap between the two companies. I unfortunately expect a lot of layoffs on the Horton side of things, which is disappointing because I know quite a few people there.
- batoure 8y agoI work for neither company but know people at both and think that this is highly inaccurate. Though there will be operational redundancy both companies make most of their revenue from support contracts and professional services since they provide a mostly FOSS product. Services and support are the two most heavily labor intensive parts of the tech industry every customer requires x amount of labor present. Additionally internal engineers paid to contribute to the FOSS products are still needed. Because both companies have been benifitting from the contributions of both teams of engineers for years. Since Horton has no closed source products it has very few redundant engineering resources.
- softliving 8y agoHmmm, merger slowdown market, employees beware of the trickery! this happens and then the mgmt thinks of redundancy to show markets how efficient they are ... is there a humane company which maybe does not pay much but secure enough and challenges us to rise together ... seems like a distant dream!
- gnanesh 8y agoLooks like MapR has got something to say to people visiting their homepage, "Two wrongs don't make a right... SEE WHY CLOUDERA AND HORTONWORKS CUSTOMERS HAVE MOVED TO MAPR!"