3 ms·

I said "you went to far" to claim Kafka, Spark and Flink do basically the same thing. It's reasonable to use all 3 of them in the same team - so they clearly do

I said "you went to far" to claim Kafka, Spark and Flink do basically the same thing. It's reasonable to use all 3 of them in the same team - so they clearly don't do the same thing. > I believe it drastically reduces adoption of these tools As does any competition. Tons of smartphone makers = less adoption for any one of them - and many will close down. Still not a bad thing.

rakoo 8y ago
> As does any competition. Those are open-source frameworks, which is antithetical to competition. The whole point of open-source is that the contributions of one entity can benefit everyone, while in this situation, having so many frameworks "dilutes" the efforts because the same problems need to be solved in each framework. Smartphone makers aren't here to share their technical advances with everyone, they're here to make money. I'm not saying we should purposefully agree on killing all but one framework, but the ideal situation is if one or two come out at the top as being just the best in their category so that everyone just use them, and all efforts converge towards those.
- virgilp 8y ago
  I don't think that's how it works; "best" is sooo hard to define. Humans are very diverse, and have different needs/preferences. Also what is best today may not be best tomorrow (not because of the system itself, but because of changes in the environment), so it pays off to have "sub-par" systems evolve in parallel with different slightly different goals or implementations. I don't see "open-source" as being antithetical to competition, at all. In fact, e.g. Databricks only exists because of Spark. There is competition in the open-source world too, and IMO that's a good thing.
- PurpleRamen 8y ago
  Since when is there no competition and diversity in open source? Isn't too much choice usually a major complain with linux, programming languages, frameworks, etc?
  - manigandham 8y ago
    Nobody said that. But having too many competing overlapping products does mean more fractured effort spread amongst all of them.
- marcinzm 8y ago
  >I'm not saying we should purposefully agree on killing all but one framework, but the ideal situation is if one or two come out at the top as being just the best in their category so that everyone just use them, and all efforts converge towards those. That assumes you can have a single framework with all the features people want without causing issues (conflicting basic designs, high cost to maintain, deployment costs, configuration costs, etc.).
- ataturk 8y ago
  If you've been around awhile no doubt you will recall the web application server wars (Spring, J2EE, Tapestry, Struts, etc.) and the JVM language wars (Java, Scala, Groovy, Clojure) and now I guess we're in the Big Data wars...

rakoo 8y ago

> As does any competition. Those are open-source frameworks, which is antithetical to competition. The whole point of open-source is that the contributions of one entity can benefit everyone, while in this situation, having so many frameworks "dilutes" the efforts because the same problems need to be solved in each framework. Smartphone makers aren't here to share their technical advances with everyone, they're here to make money. I'm not saying we should purposefully agree on killing all but one framework, but the ideal situation is if one or two come out at the top as being just the best in their category so that everyone just use them, and all efforts converge towards those.

virgilp 8y ago
I don't think that's how it works; "best" is sooo hard to define. Humans are very diverse, and have different needs/preferences. Also what is best today may not be best tomorrow (not because of the system itself, but because of changes in the environment), so it pays off to have "sub-par" systems evolve in parallel with different slightly different goals or implementations. I don't see "open-source" as being antithetical to competition, at all. In fact, e.g. Databricks only exists because of Spark. There is competition in the open-source world too, and IMO that's a good thing.
PurpleRamen 8y ago
Since when is there no competition and diversity in open source? Isn't too much choice usually a major complain with linux, programming languages, frameworks, etc?
- manigandham 8y ago
  Nobody said that. But having too many competing overlapping products does mean more fractured effort spread amongst all of them.
marcinzm 8y ago
>I'm not saying we should purposefully agree on killing all but one framework, but the ideal situation is if one or two come out at the top as being just the best in their category so that everyone just use them, and all efforts converge towards those. That assumes you can have a single framework with all the features people want without causing issues (conflicting basic designs, high cost to maintain, deployment costs, configuration costs, etc.).
ataturk 8y ago
If you've been around awhile no doubt you will recall the web application server wars (Spring, J2EE, Tapestry, Struts, etc.) and the JVM language wars (Java, Scala, Groovy, Clojure) and now I guess we're in the Big Data wars...