5 ms·
Amazon has not been contributing back to the open source projects when they offer a service based on it. E.g. https://aws.amazon.com/blogs/big-data/performance
by sg47 6y ago
Amazon has not been contributing back to the open source projects when they offer a service based on it.
E.g. https://aws.amazon.com/blogs/big-data/performance-updates-to-apache-spark-in-amazon-emr-5-24-up-to-13x-better-performance-compared-to-amazon-emr-5-16/ https://aws.amazon.com/blogs/big-data/performance-updates-to...
Amazon EMR release 5.24.0 includes several optimizations in Spark that improve query performance.
Why have these optimizations not been contributed back to the community?
- justizin 6y agobecause then you wouldn’t have been quietly walked into vendor lock-in which extends beyond the software license to the hardware you’re leasing!
- cosmodisk 6y agoBecause they don't want to, or see no value in doing so. I've been downvoted to oblivion just by stating that contributions to popular OSS projects, especially those initiated,or heavily used by large tech companies,are nothing more than just free labour. Amazon's business is commodity: they made books,one of the most precious things we,as a civilization could create,a commodity. The storage space, computing, even ML is being commoditised. Logistics will folow,then some more. The company's perspective and business goals are completely different than many would like to think.
- acatton 6y ago> The storage space, computing, even ML is being commoditised I don't see that happening anytime soon. There is still a market for hobbyist and boostrapped businesses who can't afford the insane prices offered AWS, Azure, GCP & Co. Storage, Computing and Bandwidth on these platform is inflated for the "enterprise and over-funded-startups" market. Hetzner, Ionos, Scaleway and OVH are still order of magnitude below AWS in terms of prices. And the quality difference is in no way justifying the price difference. (Especially with Hetzer and Scaleway which are almost on par with cloud platforms)
- pjmlp 6y agoBecause the community decided to use a license that allows it.
- sg47 6y agoThe community did that in good faith. What's the incentive for the community to contribute to the project if Vendor XYZ is going to offer the same project with more optimizations?
- bigiain 6y agoThe buggy whip community invested in buggy whip manufacturing too. Sometimes step changes happen, and things are just different afterwards. I see two related "changes in the world" here that are contributing to this current drama.'' 1) the rise of SaaS and globally centralised cloud hosting. 2) the emergence of companies solely (or mostly) built on paying salaries to developers writing an open source software project. (Elastic/Mongo/Redis being the primary examples). As I see it, the 30 year old "Open Source" principles didn't see these coming any more than the buggy whip industry saw the Model T coming. Personally, I'd really like to see those three companies (and others/new entrants) come up with a workable new model for software licenses for some kind of "freely usable software" ("free" in the generic "no obligations" sense, not the GPL/Stallman "freedom" sense) written by salaried devs at for-profit companies. I think the world would be a better place with more software closer to the Stallman end of the license spectrum than the Oracle end. I don't think that's necessarily a given. It might well be true that in general, a for profit company cannot make enough money out of writing software and giving it away to keep teams of full time developers paid.
- CRConrad 6y ago~20, not 30.
- jrochkind1 6y agoThe announcement claims: > All changes to Elasticsearch were sent as upstream pull requests (#42066, #42658, #43284, #43839, #53643, #57271, #59563, #61400, #64513), and we then included the “oss” builds offered by Elastic in our distribution. This ensured that we were collaborating with the upstream developers and maintainers, and not creating a “fork” of the software. With links on the issue numbers. Why should we not understand this as contributing back?
- sg47 6y agoThat's 9 PRs vs, as someone pointed out below, roughly 41000 commits in the project.
- _msw_ 6y agoDisclosure: I work at Amazon on cloud infrastructure, but not directly on the codebase being discussed here. Someone pointed out elsewhere: *Correction after looking a bit closer, I think Amazon has submitted at least 600 PRs, they only listed 9 in the blog post. That's better but it still doesn't change the fact their business model doesn't allow the companies they're building on the backs of to have a sustainable revenue stream.
- myroon5 6y agoThe other comment claiming 600 Amazon PRs was mistaken: https://news.ycombinator.com/item?id=25866527 https://news.ycombinator.com/item?id=25866527
- _msw_ 6y agoThat's counting emails. Unlike other Git-based projects like the Linux kernel, it seems a lot of commits are made with users.noreply.github.com email addresses. For example. commit 8e413f85e8978da83db107aae51e5543d4779dba Author: paulward24 <52216289+paulward24@users.noreply.github.com> Date: Wed Jul 3 04:37:07 2019 -0700 Ensure to access RecoveryState#fileDetails under lock Closes #43840
- jiofih 6y agoThere is nothing unfair about being able to make optimizations of your own on top of the software. That freedom is in fact one of the core values of OSS. Any “fairness” judgements you make are irrelevant for as long as the license satisfies its goal and the software is freely available to everyone.
- pas 6y agoFairness (ethics, moral judgement, justice) is inherently a different layer of our civilization than license (law, contracts). I'd say fairness is more important than any and all licenses ever. Also, I think through history every law was made in the name of making law more just, bringing the legal system closer to an ideal of justice. We also know that simply throwing out law and bringing back case by case judgments (eg. what kings and lords of old did) just gets us further from that ideal, hence we work on our laws and licenses. But that doesn't mean we should forget that they are imperfect and sometimes they need updating.
- jiofih 6y agoYour point being? You care more about fairness in the dealings of two billion dollar businesses, than the open in open-source?