5 ms·

Great product. It should have happened last week, then we truly could have said that all Enterprise Apache Open Software products died of the AWS plague in 2018

Great product. It should have happened last week, then we truly could have said that all Enterprise Apache Open Software products died of the AWS plague in 2018: Hadoop is gone with Cloudera/Hortonworks merging, Kafka with Confluent's new license covering their ecosystem of tooling. Databricks now have their own Spark runtime, which will get more TLC than Apache Spark. On top of Apache, Mongo, Neo4J, Redis Labs abandoning open-source. Wow, just wow.

rusk 8y ago
> On top of Apache ... abandoning open source Hold on ... what?!?
- jamesblonde 8y ago
  I think you're mis-parsing what i (hastily) wrote. On top of (these) Apache (projects being abandoned), ....
  - rusk 8y ago
    Okay that makes more sense ... you've misplaced a comma: > On top of Apache, Mongo, Neo4J, Redis Labs EDIT sorry, not the comma. Still early morning here. But you've used "Apache" as an umbrella term for "some" Apache projects. Fair enough.
sewen 8y ago
Sorry, but that is not correct. We clearly say in all announcements that this is about doubling down on open source. In contrast to all current the efforts everywhere that restrict open use, we plan to do even more work openly. This is best thing that could have happened to any open source project. And to any founder whose heart beats for technology and open source...
- jamesblonde 8y ago
  Nice comeback. Made me smile.
- jamesblonde 8y ago
  Aha, i see why you downvoted me. You were actually serious. I literally thought this was a generic tongue-in-cheek ironic comment made for the purpose of comedy. That's why it made me smile. That it's not ironic is even funnier.
  - scarejunba 8y ago
    You can’t downvote people who respond to you on HN, at least at his karma level. In fact, he couldn’t have downvoted you because he has too little karma.
monstrado 8y ago
Curious where you get "Hadoop is gone" by two companies (who employ hundreds of open source contributors and committers) merging. I think it's in fact the complete opposite. I get the whole buy-in into the FUD that Cloudera ships proprietary software so they're evil, but the fact of the matter is their core system (CDH) is 100% open source. Impala, open source. Kudu, open source. Anything that is in CDH that isn't in Apache Hadoop is in a pull request. You can literally download every last source file that you trust your data with.
- jamesblonde 8y ago
  Hadoop is not gone yet. But the intention is there. AWS EMR is killing them. Users/Companies just pick EMR by default, leaving few crumbs left for the now one company that develops the platform. The EMR team is what, a two pizza box team? Like you said, Cloudera pay for hundreds of top developers to develop a product that is repackaged by a two-pizza team whose company earns more than Cloudera on Hadoop. That situation will not continue, that is my prediction. And Hadoop will be the crossfire. My prediction is that Cloudera will release a new data platform in time that is not "hadoop", but they will leave Apache Hadoop in maintenance mode. It's the only logical way to address the AWS "problem".
  - monstrado 8y ago
    That's where we disagree. You're taking some pretty radical jumps to get to the conclusion you want. Let's assume for a second that you're right, and EMR somehow kills an entire company (who sells much more than just a Cloud offering). This has absolutely no bearing on the survival of an open source project. Yes, Cloudera employs developers who's sole job is to work on Hadoop, but there are a ton of people who work on and contribute to Hadoop outside of both Cloudera & Hortonworks (e.g. Yahoo, Facebook, Amazon, Alibaba, ..., ..., ...) It's equally outrageous to assume that the developers at these companies would somehow just completely lose all passion and willingness to carry the project forward if they lost their job. The reason a lot of people at Cloudera & Hortonworks are there is because they were contributing to the project long before they were employees. Hadoop isn't going anywhere. It may become less relevant or more relevant based on how things play out and/or new projects coming to fruition, but the project will be around for a long time. Lastly, the project "Hadoop" is nothing like it was back 8 years ago. The project continues to evolve and projects born out of it are essentially tacked onto it (e.g. Flink, Spark, Kudu, HBase, Presto/Drill/Hive/Impala).

rusk 8y ago

> On top of Apache ... abandoning open source Hold on ... what?!?

jamesblonde 8y ago
I think you're mis-parsing what i (hastily) wrote. On top of (these) Apache (projects being abandoned), ....
- rusk 8y ago
  Okay that makes more sense ... you've misplaced a comma: > On top of Apache, Mongo, Neo4J, Redis Labs EDIT sorry, not the comma. Still early morning here. But you've used "Apache" as an umbrella term for "some" Apache projects. Fair enough.

sewen 8y ago

Sorry, but that is not correct. We clearly say in all announcements that this is about doubling down on open source. In contrast to all current the efforts everywhere that restrict open use, we plan to do even more work openly. This is best thing that could have happened to any open source project. And to any founder whose heart beats for technology and open source...

jamesblonde 8y ago
Nice comeback. Made me smile.
jamesblonde 8y ago
Aha, i see why you downvoted me. You were actually serious. I literally thought this was a generic tongue-in-cheek ironic comment made for the purpose of comedy. That's why it made me smile. That it's not ironic is even funnier.
- scarejunba 8y ago
  You can’t downvote people who respond to you on HN, at least at his karma level. In fact, he couldn’t have downvoted you because he has too little karma.

monstrado 8y ago

Curious where you get "Hadoop is gone" by two companies (who employ hundreds of open source contributors and committers) merging. I think it's in fact the complete opposite. I get the whole buy-in into the FUD that Cloudera ships proprietary software so they're evil, but the fact of the matter is their core system (CDH) is 100% open source. Impala, open source. Kudu, open source. Anything that is in CDH that isn't in Apache Hadoop is in a pull request. You can literally download every last source file that you trust your data with.

jamesblonde 8y ago
Hadoop is not gone yet. But the intention is there. AWS EMR is killing them. Users/Companies just pick EMR by default, leaving few crumbs left for the now one company that develops the platform. The EMR team is what, a two pizza box team? Like you said, Cloudera pay for hundreds of top developers to develop a product that is repackaged by a two-pizza team whose company earns more than Cloudera on Hadoop. That situation will not continue, that is my prediction. And Hadoop will be the crossfire. My prediction is that Cloudera will release a new data platform in time that is not "hadoop", but they will leave Apache Hadoop in maintenance mode. It's the only logical way to address the AWS "problem".
- monstrado 8y ago
  That's where we disagree. You're taking some pretty radical jumps to get to the conclusion you want. Let's assume for a second that you're right, and EMR somehow kills an entire company (who sells much more than just a Cloud offering). This has absolutely no bearing on the survival of an open source project. Yes, Cloudera employs developers who's sole job is to work on Hadoop, but there are a ton of people who work on and contribute to Hadoop outside of both Cloudera & Hortonworks (e.g. Yahoo, Facebook, Amazon, Alibaba, ..., ..., ...) It's equally outrageous to assume that the developers at these companies would somehow just completely lose all passion and willingness to carry the project forward if they lost their job. The reason a lot of people at Cloudera & Hortonworks are there is because they were contributing to the project long before they were employees. Hadoop isn't going anywhere. It may become less relevant or more relevant based on how things play out and/or new projects coming to fruition, but the project will be around for a long time. Lastly, the project "Hadoop" is nothing like it was back 8 years ago. The project continues to evolve and projects born out of it are essentially tacked onto it (e.g. Flink, Spark, Kudu, HBase, Presto/Drill/Hive/Impala).