5 ms·

As an active developer and maintainer of a cluster, I strongly disapprove of this move by AWS. By forking on an earlier instance of the codebase for ES and Kib

As an active developer and maintainer of a cluster, I strongly disapprove of this move by AWS. By forking on an earlier instance of the codebase for ES and Kibana they're not only creating an open source version of the code but also attempting a fork of the community - plugin developers, user groups, etc. It's extremely frustrating. EDIT: If folks actively building plugins have felt pain here because of decisions by Elastic, then my view on this point changes considerably, obviously. I find it very hard to believe there was no compromise to be had here between the current Elastic roadmap for their product and their previous work. How is this good for users of the current tools? How does this help create better capabilities on a common platform? Will this encourage users to build businesses on elasticsearch open source elements? I'm not sure where the community ends up after this, so I'm not sure I can support by using AWS' elasticsearch tools.

sidi 8y ago
As someone who is involved in the ecosystem and maintains some open-source projects based on ElasticSearch, I am similarly curious about the move to not build upon existing projects (there could be a good reason, but haven't heard it yet). Why wasn't ElastAlert used / built upon for alerting? https://github.com/yelp/elastalert https://github.com/yelp/elastalert Or why wasn't ElasticSearch SQL used / built upon for SQL? https://github.com/NLPchina/elasticsearch-sql https://github.com/NLPchina/elasticsearch-sql Or why wasn't SearchGuard used / built upon for Security? https://github.com/floragunncom/search-guard https://github.com/floragunncom/search-guard All of them have Apache 2.0 license.
- ssgao 8y ago
  For SearchGuard, only the basic auth is Apache 2.0. Anything practical you need to pay to use it.
  - sagichmal 8y ago
    Why isn't AWS paying to use it, then? It's not like they don't have the money?
    - ssgao 8y ago
      there's no "they", there's only us. AWS paying to use it means we are paying to use it. Between that and a truly Apache 2.0 project, I think it's a no-brainer for everyone.
      - sagichmal 8y ago
        Why shouldn't you pay to use it, then? More generally: why should AWS get to leverage its near-monopolistic position in the SaaS market to perform essentially hostile takeovers of the software projects it packages and runs? Simply because they have the resources to do it?
        e1ven 8y ago
        I'm very glad that they've done this. The code was open, and the company decided to close it for the new versions, so it forks. Forks are common when people don't like the corporate direction of an OSS project - It's a strength, not a weakness. Openoffice/Libreoffice, Hudson/Jenkins etc. Imagine if Linus said that the next version of Linux would be paid-only. Within 15 minutes there'd be a new librelinux repo that people could contribute to instead. Someone was going to make a new fork anyway - This is Amazon putting their money where their mouth is to fund and support that new fork.
        dhd415 8y ago
        To be clear, no Elasticsearch code or features that were OSS were closed. As new Elasticsearch features are developed, some of them are now released as OSS and others are released with a commercial license.
        anth_anm 8y ago
        Okay, are you glad that Amazon now owns Elasticsearch?
        therealx 8y ago
        If ES didn't do the weird license change thing, we wouldn't be here. Either Amazon wouldn't have used it, or they'd be paying. Including non-com features in the default install was not a good way to play things. Would you feel the same if AWS built a drop in/api compatible system from scratch?
  - termez442 8y ago
    From a look at the source code, Amazon has Apached 2.0 the entire Floragunn Searchguard codebase (including the commercial part)
- edraferi 8y ago
  > why wasn't ElasticSearch SQL used / built upon for SQL? According to the release blogpost [0], "SQL Support [...]is an improved version of the elasticsearch-sql plugin." Sounds like they are building on it, as you suggested. [0] https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic.. https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic....
- stoke 8y ago
  > Or why wasn't SearchGuard used / built upon for Security? https://github.com/floragunncom/search-guard https://github.com/floragunncom/search-guard Looks like it was. https://github.com/opendistro-for-elasticsearch/security-ssl/blob/master/NOTICE.txt https://github.com/opendistro-for-elasticsearch/security-ssl... floragunn GmbH Copyright 2015-2018 floragunn GmbH
- unethical_ban 8y ago
  Beyond the thoughtful ideas below, an official blog post would be cool for that. Specifically, Elastalert and ODfE Alerting are completely different.
MeteorMarc 8y ago
From the pom.xml of Open Distro for ElasticSearch: the fork is from elasticsearch-6.5.4, which is the latest stable release. Only alternative would have been v6.6.1. EDIT: also, nothing is there to believe they would not bump to higher versions in later versions.
- praseodym 8y ago
  6.6.1 is the latest stable release. 6.5.4 was released 19 december 2018 [1], so they’re not too far behind, though. [1] https://www.elastic.co/blog/elastic-stack-6-5-4-released https://www.elastic.co/blog/elastic-stack-6-5-4-released
- sailfast 8y ago
  Thanks for this clarification!
SeaCow 8y ago
From the related AWS blog (so at least a few grains of salt): >>This means that the majority of new Elasticsearch users are now, in fact, running proprietary software. We have discussed our concerns with Elastic, the maintainers of Elasticsearch, including offering to dedicate significant resources to help support a community-driven, non-intermingled version of Elasticsearch. They have made it clear that they intend to continue on their current path. ... maybe I'm misreading your comment, but it sounds like AWS tried to find a compromise, including contributing to upstream, and were shut down (and all their content about future contributing seems to say "we'll send it upstream, but it depends on whether Elastic accepts it," so seems to corroborate). It honestly seems like you get more certainty that everything you're using from AWS is open and will remain open, and Elastic is going to keep creating uncertainty / drive to proprietary source-available.
- sailfast 8y ago
  There is certainty around the open-source bits and the license - this is true. That said elastic the company has also seems to have brought the product a long way after becoming a company. Having a dedicated staff of engineers and developers behind a thing is helpful for that, and they should be able to make some money as part of this if we want to see future open source products advance similarly. (There is a difference too between consulting / services and money “at scale” - AWS is removing an at-scale component here) That said, I also totally understand disagreements with Elastic’s decisions on charging for security, etc and why that causes concern. As a user I wish I wasn’t staring a fork in the face and I’m hopeful things get back to equilibrium here soon as a single community.

sidi 8y ago

As someone who is involved in the ecosystem and maintains some open-source projects based on ElasticSearch, I am similarly curious about the move to not build upon existing projects (there could be a good reason, but haven't heard it yet). Why wasn't ElastAlert used / built upon for alerting? https://github.com/yelp/elastalert https://github.com/yelp/elastalert Or why wasn't ElasticSearch SQL used / built upon for SQL? https://github.com/NLPchina/elasticsearch-sql https://github.com/NLPchina/elasticsearch-sql Or why wasn't SearchGuard used / built upon for Security? https://github.com/floragunncom/search-guard https://github.com/floragunncom/search-guard All of them have Apache 2.0 license.

ssgao 8y ago
For SearchGuard, only the basic auth is Apache 2.0. Anything practical you need to pay to use it.
- sagichmal 8y ago
  Why isn't AWS paying to use it, then? It's not like they don't have the money?
  - ssgao 8y ago
    there's no "they", there's only us. AWS paying to use it means we are paying to use it. Between that and a truly Apache 2.0 project, I think it's a no-brainer for everyone.
    - sagichmal 8y ago
      Why shouldn't you pay to use it, then? More generally: why should AWS get to leverage its near-monopolistic position in the SaaS market to perform essentially hostile takeovers of the software projects it packages and runs? Simply because they have the resources to do it?
      - e1ven 8y ago
        I'm very glad that they've done this. The code was open, and the company decided to close it for the new versions, so it forks. Forks are common when people don't like the corporate direction of an OSS project - It's a strength, not a weakness. Openoffice/Libreoffice, Hudson/Jenkins etc. Imagine if Linus said that the next version of Linux would be paid-only. Within 15 minutes there'd be a new librelinux repo that people could contribute to instead. Someone was going to make a new fork anyway - This is Amazon putting their money where their mouth is to fund and support that new fork.
        dhd415 8y ago
        To be clear, no Elasticsearch code or features that were OSS were closed. As new Elasticsearch features are developed, some of them are now released as OSS and others are released with a commercial license.
        anth_anm 8y ago
        Okay, are you glad that Amazon now owns Elasticsearch?
      - therealx 8y ago
        If ES didn't do the weird license change thing, we wouldn't be here. Either Amazon wouldn't have used it, or they'd be paying. Including non-com features in the default install was not a good way to play things. Would you feel the same if AWS built a drop in/api compatible system from scratch?
- termez442 8y ago
  From a look at the source code, Amazon has Apached 2.0 the entire Floragunn Searchguard codebase (including the commercial part)
edraferi 8y ago
> why wasn't ElasticSearch SQL used / built upon for SQL? According to the release blogpost [0], "SQL Support [...]is an improved version of the elasticsearch-sql plugin." Sounds like they are building on it, as you suggested. [0] https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic.. https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic....
stoke 8y ago
> Or why wasn't SearchGuard used / built upon for Security? https://github.com/floragunncom/search-guard https://github.com/floragunncom/search-guard Looks like it was. https://github.com/opendistro-for-elasticsearch/security-ssl/blob/master/NOTICE.txt https://github.com/opendistro-for-elasticsearch/security-ssl... floragunn GmbH Copyright 2015-2018 floragunn GmbH
unethical_ban 8y ago
Beyond the thoughtful ideas below, an official blog post would be cool for that. Specifically, Elastalert and ODfE Alerting are completely different.

MeteorMarc 8y ago

From the pom.xml of Open Distro for ElasticSearch: the fork is from elasticsearch-6.5.4, which is the latest stable release. Only alternative would have been v6.6.1. EDIT: also, nothing is there to believe they would not bump to higher versions in later versions.

praseodym 8y ago
6.6.1 is the latest stable release. 6.5.4 was released 19 december 2018 [1], so they’re not too far behind, though. [1] https://www.elastic.co/blog/elastic-stack-6-5-4-released https://www.elastic.co/blog/elastic-stack-6-5-4-released
sailfast 8y ago
Thanks for this clarification!

SeaCow 8y ago

From the related AWS blog (so at least a few grains of salt): >>This means that the majority of new Elasticsearch users are now, in fact, running proprietary software. We have discussed our concerns with Elastic, the maintainers of Elasticsearch, including offering to dedicate significant resources to help support a community-driven, non-intermingled version of Elasticsearch. They have made it clear that they intend to continue on their current path. ... maybe I'm misreading your comment, but it sounds like AWS tried to find a compromise, including contributing to upstream, and were shut down (and all their content about future contributing seems to say "we'll send it upstream, but it depends on whether Elastic accepts it," so seems to corroborate). It honestly seems like you get more certainty that everything you're using from AWS is open and will remain open, and Elastic is going to keep creating uncertainty / drive to proprietary source-available.

sailfast 8y ago
There is certainty around the open-source bits and the license - this is true. That said elastic the company has also seems to have brought the product a long way after becoming a company. Having a dedicated staff of engineers and developers behind a thing is helpful for that, and they should be able to make some money as part of this if we want to see future open source products advance similarly. (There is a difference too between consulting / services and money “at scale” - AWS is removing an at-scale component here) That said, I also totally understand disagreements with Elastic’s decisions on charging for security, etc and why that causes concern. As a user I wish I wasn’t staring a fork in the face and I’m hopeful things get back to equilibrium here soon as a single community.