4 ms·
EMR isn't open source but Spark is. What does the EMR Spark Runtime if not offer Spark as a service? And the changes to optimize spark runtime, why were they no
by sg47 7y ago
EMR isn't open source but Spark is. What does the EMR Spark Runtime if not offer Spark as a service? And the changes to optimize spark runtime, why were they not contributed back to upstream Spark?
- manigandham 7y agoI'm sure some changes are upstreamed (assuming they're even accepted) but there's no requirement for them to do so. Again this seems like a random example. What is so important about this particular change over all the other open-source contributions?
- sg47 7y agoThis is just an example. I'm sure there are many others. The developers that take the time to contribute to Spark are making Spark a better product. Amazon is not making it better. Amazon should not claim they made improvements to Spark in a newer version. What they did was upgrade Spark to 2.4 and claim that the improvements were done by them whereas in reality they were done by the community.
- manigandham 7y agoAgain this is all wild assumption. They're not required to contribute to free software, but they do upstream changes. The improvements they're claiming is for their own EMR product, not to Spark, and they do make updates to software to run better on their own own infrastructure. That's what their customers care about. You seem dedicated to believing that Amazon has done nothing for open-source even I pointed you to an exhaustive list of their contributions. At this point, there's nothing more to say.