14 ms·

Official Elasticsearch Python library no longer works with open-source forks

294 points by maciekmm 5y ago

make3 5y ago
that sucks.
catmanjan 5y ago
Can't Amazon just serve these bogus headers and keep opensearch compatible? It's not like headers are copyrighted...
- evil-olive 5y ago
  Amazon could try that, but it wouldn't stop there. they'd end up in a constant arms race. also, that "you know, for search" tagline is featured on t-shirts sold by Elastic [0]. seems entirely plausible to me that Amazon & Elastic lawyers could get into a pissing contest about potential trademark or copyright protection of that term and whether sending it as an HTTP header constitutes infringement. 0: https://elastic.shop/products/you-know-for-search-t-shirt-straight-cut https://elastic.shop/products/you-know-for-search-t-shirt-st...
  - contravariant 5y ago
    Though I suppose they're now implicitly making it part of the API, which I think was ruled to fall under fair-use, though I'm not too sure about the details.
  - easton 5y ago
    “You Know, for Search” is sent in Elasticsearch 6.8 on AWS. Was it removed at some point?
  - javagram 5y ago
    Possible precedent for the use of copyright law in such a way: https://www.eff.org/cases/lexmark-v-static-control-case-archive https://www.eff.org/cases/lexmark-v-static-control-case-arch...
  - toast0 5y ago
    I'm sure the lawyers would be happy to spent lots of time hashing out the details, but Sega v Accolade [1] would seem to be relevant and allow Amazon to use trademarks for interoperability. [1] https://en.m.wikipedia.org/wiki/Sega_v._Accolade https://en.m.wikipedia.org/wiki/Sega_v._Accolade
  - inet6 5y ago
    Apple won their Hackintosh copyright lawsuit against Psystar over their copyright haiku that the machine will refuse to boot if absent. https://www.zdnet.com/article/apple-warns-off-os-pirates-with-a-poem/ https://www.zdnet.com/article/apple-warns-off-os-pirates-wit...
    - easton 5y ago
      The haiku is “ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc”, whereas the linked poem is loaded with DSMOS I think.
- contravariant 5y ago
  No need for the headers just return a 401 or 403 on a GET to '/'. > If call to '/' fails with 401 or 403 pass the check and show a warning (message will be linked later). This happens if the monitor permission missing for user. The subsequent checks must be ignored.
- fragmede 5y ago
  Likely! It worked against SEGA, in the Sega v. Accolade case. Unlicensed Accolade (and later, SEGA Dreamcast homebrew games) displayed a Sega copyright logo because they were (believed to be) needed to run arbitrary executables.
  - EvanAnderson 5y ago
    Makes me think of "Oracle poetry": https://dacut.blogspot.com/2008/03/oracle-poetry.html https://dacut.blogspot.com/2008/03/oracle-poetry.html
    - cerved 5y ago
      wow
    - int_19h 5y ago
      So the idea is that a poem, being a work of art, is actually copyrightable? Does this legal hack work?
      - EvanAnderson 5y ago
        The finding in Sega v. Accolade would seem to exclude this Oracle 'hack', too, as being fair use: https://www.copyright.gov/fair-use/summaries/segaenters-accolade-9thcir1992.pdf https://www.copyright.gov/fair-use/summaries/segaenters-acco...
  - djmips 5y ago
    As a side note, although Accolade won this case, the injunction that SEGA won earlier prevented Accolade from selling product for a while and that cash flow problem motivated their primary investor to step in and remove the company's founder and all top executives and bring in their own team which directly led to the end of Accolade as a going concern (my opinion). Of course in this case, Amazon isn't as vulnerable as Accolade was back then.
- leros 5y ago
  Amazon is doing the opposite. They're making their OpenSearch client compatible with both OpenSearch and ElasticSearch.
- deleted 5y ago
  [deleted]
thedougd 5y ago
I use to be a huge proponent of Elastic.co for search. I paid for their Elastic Cloud Enterprise product to offer an internally hosted Elastic as a Service. We ran a proof of concept with the the open source Elastic Stack before requesting the funding to get commercial support and the extra management features. I'm not sure I can recommend them anymore. The company doesn't seem to understand their customer's journey and are rejecting the methods that brought them success.
- echelon 5y ago
  It's hard to blame them when Amazon is eating their lunch. Well, it's more like they made a sandwich, Amazon stole it, and now Amazon is selling it to other people. Meanwhile Elastic.co is starving. It's not a fair game anymore. I hope the DOJ breaks them up into three or more companies.
  - __blockcipher__ 5y ago
    This is a false narrative. Both Elastic and Amazon are making a killing. > Elastic (NYSE: ESTC) ("Elastic"), the company behind Elasticsearch and the Elastic Stack, announced strong results for its fourth quarter and full fiscal year (ended April 30, 2021). Total revenue was $177.6 million, an increase of 44% year-over-year, or 39% on a constant currency basis.
    - jimmydorry 5y ago
      Revenue only tells us how much business they conducted, not how much of a "killing" they made. Profits is what would tell you that, which I guarentee are going to be far less than their $177.6 million in revenue.
      - ethbr0 5y ago
        You don't have to guess: they're a public company. https://finance.yahoo.com/quote/ESTC/financials?p=ESTC https://finance.yahoo.com/quote/ESTC/financials?p=ESTC
orf 5y ago
What differences cause issues?
atsaloli 5y ago
See also https://github.com/elastic/elasticsearch-py/issues/1666 https://github.com/elastic/elasticsearch-py/issues/1666
whalesalad 5y ago
PostgreSQL should be everyone’s first choice for a data store. It can do so much, including serving as your full text search system.
- ageitgey 5y ago
  I love postgres and the full-text search feature works great in some use cases, but it is not really comparable to elastic search in many scenarios (huge document stores, complex text processing or querying, etc).
  - catmanjan 5y ago
    Do you know for sure that postgres doesn't perform as well as elasticsearch if you don't use the relational capabilities of postgres? Instinctively I believe what you're saying, just wondering if you know for sure.
    - MapleWalnut 5y ago
      Scoring results in Postgres requires scanning all matches, which is slow if you have a lot of results. Elastic search and other search solutions don’t have this problem.
    - rpedela 5y ago
      Yes I know for sure. Postgres search is essentially an easier to use regex engine. If you have a recall-only use case and/or a small dataset, then that works great. As soon as you need multiple languages, advanced autocomplete, misspelling detection, large documents, large datasets, custom scoring, etc you need Solr or ES.
      - hardwaresofton 5y ago
        While I don't doubt that you know your usecase and weighed/tried the option. > Postgres search is essentially an easier to use regex engine. I'm not sure exactly what you meant to convey here, but if you're searching with LIKE or `~` you're not doing Postgres's proper Full Text Search. You should be dealing with tsvectors[0] > As soon as you need multiple languages Postgres FTS supports multiple languages and you can create your own configurations[1] > advanced autocomplete I'm not sure what "advanced" autocomplete is but you can get pretty fast trigram searches going[2] (back to LIKE/ILIKE here but obviously this is an isolated usecase). In the end I'd expect auto complete results to actually not hit your DB most of the time (maybe I'm naive but that feels like a caching > cache invalidation > cache pushdown problem to me) > misspelling detection pg_similarity_extension[3] might be of some help here, but it may require some wrangling. > large documents, large datasets, PG has TOAST[4], and obviously can scale (maybe not necessarily great at it) -- see pg_partman/Timescale/Citus/etc. > custom scoring Postgres only has basic ranking features[5], but you can write your own functions and extend it of course. Solr/ES are definitely the right tools for the job (tm) when the job is search, but you can get surprisingly far with Postgres. I'd argue that many usecases actually don't want/need a perfect full text search solution -- it's often minor features that turn into overkill fests and ops people learning/figuring out how to properly manage and scale an ES cluster and falling into pitfalls along the way. [0]: https://www.postgresql.org/docs/current/textsearch-intro.html#TEXTSEARCH-DOCUMENT https://www.postgresql.org/docs/current/textsearch-intro.htm... [1]: https://www.postgresql.org/docs/current/textsearch-intro.html#TEXTSEARCH-INTRO-CONFIGURATIONS https://www.postgresql.org/docs/current/textsearch-intro.htm... [2]: https://about.gitlab.com/blog/2016/03/18/fast-search-using-postgresql-trigram-indexes/ https://about.gitlab.com/blog/2016/03/18/fast-search-using-p... [3]: https://github.com/eulerto/pg_similarity https://github.com/eulerto/pg_similarity [4]: https://www.postgresql.org/docs/current/storage-toast.html https://www.postgresql.org/docs/current/storage-toast.html [5]: https://www.postgresql.org/docs/9.5/textsearch-controls.html#TEXTSEARCH-RANKING https://www.postgresql.org/docs/9.5/textsearch-controls.html...
karmakaze 5y ago
This should be forked to be opensource compatible. Opensource doesn't mean that you get to control everything. If what you're offering doesn't stand on its own merits then what's it worth? I get that we wish Amazon couldn't do what they do but that's what the license permits.
- TAForObvReasons 5y ago
  The maintainers of the project chose to implement this change. You may not like it, and you are free to make a fork compatible with the original apache 2.0 license, but it is well within Elastic's right to add this. This is the danger with corporate open source in general. Facebook made a similar unilateral decision in relicensing React from Apache 2 to "BSD + Patents" many years ago. They faced quite a bit of pressure, resulting in another relicensing to MIT.
  - geofft 5y ago
    No one is disputing it's within their rights. It's within their rights to add a feature to the library that blasts "Caramelldansen" through the nearest Chromecast device when you import it, too. What they're saying is that this is a breach of community norms.
  - lolinder 5y ago
    They have the right to do so, but that doesn't change the fact that it's a bad idea politically. It comes off as Elastic prioritizing their war with Amazon over the community, leaving room for Amazon to continue to position themselves as the community's ally.
    - altdataseller 5y ago
      At the end of the day, CTOs and decision makers in corporations don't go with the company that position themselves as "allies". They go with the company that is shipping important new features, with better support and resiliency. If Elastic is shipping features faster than AWS is, then they'll go with Elastic. I bet most CTOs aren't even aware of the politics happening - it's only the echo chamber in Hacker News that is making a big deal of this.
MilnerRoute 5y ago
Hacker News had a good discussion about this earlier today. https://news.ycombinator.com/item?id=28103389 https://news.ycombinator.com/item?id=28103389 What surprised me was how many commenters seemed unhappy with Amazon's Open Source alternative and the level of resources they appeared to be committing to it.
- thedougd 5y ago
  Amazon is certainly the big bad wolf here, but I wouldn't be surprised if the move is also designed to inconvenience the small players like logz.io, Graylog, Searchly, etc who might be interfering with their flow of introductory level customers. A quick search reveals there are more of these providers than I ever realized.
- marcinzm 5y ago
  AWS is pretty well known for rolling out half baked products with understaffed teams especially when it's a re-hosting of an open source product. EKS was a mess for a long time and had a tiny team behind it from what I hear.
deleted 5y ago
[deleted]
iandanforth 5y ago
Good? Amazon deserves no support in branding their version 'open'.
- Spivak 5y ago
  You do realize that Amazon would have been happy selling hosting for the official Elastic code until Elastic forced them into a situation where the pull the rug out of all of their customers or fork? Amazon is actually the more open company when it comes to Elastic at this point. Open source does best when it’s not the meal ticket of the company developing it.
  - nijave 5y ago
    It's a bit clunky, but Amazon's fork also comes with security by default. ES open-by-default has caused breaches for years. Amazon's is a bit clunky to setup but at least comes with defaults where you are less likely to leak your data
egberts 5y ago
It’s nigh time to fork the ElasticSearch API.
reilly3000 5y ago
I’m not going to say that Elastic has no real competition, but as a whole package it stands alone. It’s speed and versatility definitely differentiate it from other similar systems. The fact that it can be a log aggregator, search engine, personalization system, seim, analytics tool, and forecasting tool make it a lot more useful than typical full-text search systems, often which can’t process aggregations or run in a single instance. I think they know their position in the market is strong and that helps them feel empowered to make risky moves with their licensing. I’d rather use anything else, but for interactive analytics (sub second response times on millions of data points) it’s tough to beat. It’s a fussy app to self-host or run in a non-managed cloud environment. I wish them the best, but worry these kinds of moves will inspire more competition from totally other projects, let alone it’s fork(s).
- WontonDon 5y ago
  As long as they keep innovating to stay number 1, they will do just fine, even against AWS fork or new competitors.
- nullify88 5y ago
  A lot of people like myself just want to view logs and dont care for features like SIEM. This is where the OSS and OpenDistro versions shine. I have revisited Loki after all this new though, but I think its still missing full text search.
  - chippiewill 5y ago
    > but I think its still missing full text search. That's very much by design, by not having indexing of the logging content it's much much leaner and efficient. It's aimed at a completely different use case where you already know the set of log streams you want to monitor.
teclordphrack2 5y ago
This is definitaly going to blow up in their face. I know at least a dozen people who learned open source elastic search in a research environment and then moved jobs to doing it in a for profit company.If the paid vs open source products are not going to be the same then your losing training that companies want. I also have ran into the scenario that others have described where we use the open source version for proof of concept b/4 moving to the paid version when needed. At the least, this muddies the waters and creates extra research time for engineers just to understand the licensing and cost implications of the product.
mattmcknight 5y ago
People who want open source search should take another look at Solr, it's doing fine.
- mdaniel 5y ago
  I don't exactly know why you're getting downvoted, but I will point out that while they play in the same sandbox, this is roughly equivalent to suggesting that someone rewrite their app because Solr and ES have absolutely stellar differences of opinion about the API used to access them, the amount of schema one must provide up-front to the document index, and the amount of operational hoops one must jump through to keep the service alive Sure, there are about 5-6 open source search engines one could pick from in a greenfield project, with amazing differences of features and maturity, but choosing between Open Search and ElasticSearch is likely a "political" decision, since they are (ahem, mostly) API compatible and have very similar operational needs.
Proven 5y ago
Why would they waste their time testing and maintaining it for the forks? Those other forks are (alsoA) open source, what's the problem? Maintain your own library if you need it.
kelnage 5y ago
Elastic, like others at the time, have used open source to their advantage - to start. The obvious one is that it is built around the Apache Lucene, and I have no doubt that this is one of the reasons ES ended up being initially released under the Apache license. Secondly, being released under a permissive open source license definitely helped with its adoption. I was working as a senior developer in a UK Government department in 2013 and we had need for a full-text search engine for a project - and even before v1, ES was a contender, and it was eventually selected once v1 was released. This was largely due to a) ease of set-up/use and b) it’s release under an open source license. If it had been under AGPL, would we have still used it? Yes, probably - our specific use case wouldn’t have been affected by such a license, and the dept. was relatively open to more complex OSS licenses - but I have worked for several other orgs. where even just the AGPL would have resulted in a hard “no”. Thirdly, ES has had contributions from a wide range of people. I honestly don’t know how I’d even begin to evaluate how much value ES has got from community contributions, but I feel it’s likely to be greater than the costs of managing those contributions. But of course eventually Elastic got funding and had shareholders to placate. I don’t really have much sympathy for them about this conflict - their early choices were in part clearly made to maximise their value, and they decided to cash in on that value at a later date - the fact that those decisions had implications for their value should have been somewhat obvious to any investor who did any due diligence. I’m still not entirely convinced that the open source model is antithetical to commercialisation, but I think it does highlight how early decisions around OSS licensing can affect such processes.
- jka 5y ago
  I keep hearing the AGPL-makes-corporations-wince argument, and I'm curious: what are the reasons given? Does the AGPL really place such burdens on the organization such that the benefits of a locked-open, community-guaranteed (albeit popularity not guaranteed) technology aren't worthwhile? Or is it kind of a cargo-culting and cultural-norms phenomenon where people don't use AGPL projects because they've heard that other people don't use them, thus continuing the cycle?
  - kelnage 5y ago
    In one of the of the organisations I mentioned, they had a strict policy against using any GPL dependencies, let alone the AGPL. I tried discussing this with the legal policy person but they were quite resolute - they feared it’s use could “infect” our code and therefore must be avoided. I frankly doubt there’s any sort of cost-benefit analysis being done here. Certainly in my experience it was much more driven by legal uncertainty and risk-aversion.
PeterZaitsev 5y ago
Oh my. I'm not sure if there is any Open Source company which fell so low - to Break the Drivers... I'm also not sure what they are looking to reach with this step as Developers who want their applications to work both with Elastic and OpenSearch will not need to go right into AWS hands...

294 points by maciekmm 5y ago

make3 5y ago

that sucks.

catmanjan 5y ago

Can't Amazon just serve these bogus headers and keep opensearch compatible? It's not like headers are copyrighted...

evil-olive 5y ago
Amazon could try that, but it wouldn't stop there. they'd end up in a constant arms race. also, that "you know, for search" tagline is featured on t-shirts sold by Elastic [0]. seems entirely plausible to me that Amazon & Elastic lawyers could get into a pissing contest about potential trademark or copyright protection of that term and whether sending it as an HTTP header constitutes infringement. 0: https://elastic.shop/products/you-know-for-search-t-shirt-straight-cut https://elastic.shop/products/you-know-for-search-t-shirt-st...
- contravariant 5y ago
  Though I suppose they're now implicitly making it part of the API, which I think was ruled to fall under fair-use, though I'm not too sure about the details.
- easton 5y ago
  “You Know, for Search” is sent in Elasticsearch 6.8 on AWS. Was it removed at some point?
- javagram 5y ago
  Possible precedent for the use of copyright law in such a way: https://www.eff.org/cases/lexmark-v-static-control-case-archive https://www.eff.org/cases/lexmark-v-static-control-case-arch...
- toast0 5y ago
  I'm sure the lawyers would be happy to spent lots of time hashing out the details, but Sega v Accolade [1] would seem to be relevant and allow Amazon to use trademarks for interoperability. [1] https://en.m.wikipedia.org/wiki/Sega_v._Accolade https://en.m.wikipedia.org/wiki/Sega_v._Accolade
- inet6 5y ago
  Apple won their Hackintosh copyright lawsuit against Psystar over their copyright haiku that the machine will refuse to boot if absent. https://www.zdnet.com/article/apple-warns-off-os-pirates-with-a-poem/ https://www.zdnet.com/article/apple-warns-off-os-pirates-wit...
  - easton 5y ago
    The haiku is “ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc”, whereas the linked poem is loaded with DSMOS I think.
contravariant 5y ago
No need for the headers just return a 401 or 403 on a GET to '/'. > If call to '/' fails with 401 or 403 pass the check and show a warning (message will be linked later). This happens if the monitor permission missing for user. The subsequent checks must be ignored.
fragmede 5y ago
Likely! It worked against SEGA, in the Sega v. Accolade case. Unlicensed Accolade (and later, SEGA Dreamcast homebrew games) displayed a Sega copyright logo because they were (believed to be) needed to run arbitrary executables.
- EvanAnderson 5y ago
  Makes me think of "Oracle poetry": https://dacut.blogspot.com/2008/03/oracle-poetry.html https://dacut.blogspot.com/2008/03/oracle-poetry.html
  - cerved 5y ago
    wow
  - int_19h 5y ago
    So the idea is that a poem, being a work of art, is actually copyrightable? Does this legal hack work?
    - EvanAnderson 5y ago
      The finding in Sega v. Accolade would seem to exclude this Oracle 'hack', too, as being fair use: https://www.copyright.gov/fair-use/summaries/segaenters-accolade-9thcir1992.pdf https://www.copyright.gov/fair-use/summaries/segaenters-acco...
- djmips 5y ago
  As a side note, although Accolade won this case, the injunction that SEGA won earlier prevented Accolade from selling product for a while and that cash flow problem motivated their primary investor to step in and remove the company's founder and all top executives and bring in their own team which directly led to the end of Accolade as a going concern (my opinion). Of course in this case, Amazon isn't as vulnerable as Accolade was back then.
leros 5y ago
Amazon is doing the opposite. They're making their OpenSearch client compatible with both OpenSearch and ElasticSearch.
deleted 5y ago
[deleted]

thedougd 5y ago

I use to be a huge proponent of Elastic.co for search. I paid for their Elastic Cloud Enterprise product to offer an internally hosted Elastic as a Service. We ran a proof of concept with the the open source Elastic Stack before requesting the funding to get commercial support and the extra management features. I'm not sure I can recommend them anymore. The company doesn't seem to understand their customer's journey and are rejecting the methods that brought them success.

echelon 5y ago
It's hard to blame them when Amazon is eating their lunch. Well, it's more like they made a sandwich, Amazon stole it, and now Amazon is selling it to other people. Meanwhile Elastic.co is starving. It's not a fair game anymore. I hope the DOJ breaks them up into three or more companies.
- __blockcipher__ 5y ago
  This is a false narrative. Both Elastic and Amazon are making a killing. > Elastic (NYSE: ESTC) ("Elastic"), the company behind Elasticsearch and the Elastic Stack, announced strong results for its fourth quarter and full fiscal year (ended April 30, 2021). Total revenue was $177.6 million, an increase of 44% year-over-year, or 39% on a constant currency basis.
  - jimmydorry 5y ago
    Revenue only tells us how much business they conducted, not how much of a "killing" they made. Profits is what would tell you that, which I guarentee are going to be far less than their $177.6 million in revenue.
    - ethbr0 5y ago
      You don't have to guess: they're a public company. https://finance.yahoo.com/quote/ESTC/financials?p=ESTC https://finance.yahoo.com/quote/ESTC/financials?p=ESTC

orf 5y ago

What differences cause issues?

atsaloli 5y ago

See also https://github.com/elastic/elasticsearch-py/issues/1666 https://github.com/elastic/elasticsearch-py/issues/1666

whalesalad 5y ago

PostgreSQL should be everyone’s first choice for a data store. It can do so much, including serving as your full text search system.

ageitgey 5y ago
I love postgres and the full-text search feature works great in some use cases, but it is not really comparable to elastic search in many scenarios (huge document stores, complex text processing or querying, etc).
- catmanjan 5y ago
  Do you know for sure that postgres doesn't perform as well as elasticsearch if you don't use the relational capabilities of postgres? Instinctively I believe what you're saying, just wondering if you know for sure.
  - MapleWalnut 5y ago
    Scoring results in Postgres requires scanning all matches, which is slow if you have a lot of results. Elastic search and other search solutions don’t have this problem.
  - rpedela 5y ago
    Yes I know for sure. Postgres search is essentially an easier to use regex engine. If you have a recall-only use case and/or a small dataset, then that works great. As soon as you need multiple languages, advanced autocomplete, misspelling detection, large documents, large datasets, custom scoring, etc you need Solr or ES.
    - hardwaresofton 5y ago
      While I don't doubt that you know your usecase and weighed/tried the option. > Postgres search is essentially an easier to use regex engine. I'm not sure exactly what you meant to convey here, but if you're searching with LIKE or `~` you're not doing Postgres's proper Full Text Search. You should be dealing with tsvectors[0] > As soon as you need multiple languages Postgres FTS supports multiple languages and you can create your own configurations[1] > advanced autocomplete I'm not sure what "advanced" autocomplete is but you can get pretty fast trigram searches going[2] (back to LIKE/ILIKE here but obviously this is an isolated usecase). In the end I'd expect auto complete results to actually not hit your DB most of the time (maybe I'm naive but that feels like a caching > cache invalidation > cache pushdown problem to me) > misspelling detection pg_similarity_extension[3] might be of some help here, but it may require some wrangling. > large documents, large datasets, PG has TOAST[4], and obviously can scale (maybe not necessarily great at it) -- see pg_partman/Timescale/Citus/etc. > custom scoring Postgres only has basic ranking features[5], but you can write your own functions and extend it of course. Solr/ES are definitely the right tools for the job (tm) when the job is search, but you can get surprisingly far with Postgres. I'd argue that many usecases actually don't want/need a perfect full text search solution -- it's often minor features that turn into overkill fests and ops people learning/figuring out how to properly manage and scale an ES cluster and falling into pitfalls along the way. [0]: https://www.postgresql.org/docs/current/textsearch-intro.html#TEXTSEARCH-DOCUMENT https://www.postgresql.org/docs/current/textsearch-intro.htm... [1]: https://www.postgresql.org/docs/current/textsearch-intro.html#TEXTSEARCH-INTRO-CONFIGURATIONS https://www.postgresql.org/docs/current/textsearch-intro.htm... [2]: https://about.gitlab.com/blog/2016/03/18/fast-search-using-postgresql-trigram-indexes/ https://about.gitlab.com/blog/2016/03/18/fast-search-using-p... [3]: https://github.com/eulerto/pg_similarity https://github.com/eulerto/pg_similarity [4]: https://www.postgresql.org/docs/current/storage-toast.html https://www.postgresql.org/docs/current/storage-toast.html [5]: https://www.postgresql.org/docs/9.5/textsearch-controls.html#TEXTSEARCH-RANKING https://www.postgresql.org/docs/9.5/textsearch-controls.html...

karmakaze 5y ago

This should be forked to be opensource compatible. Opensource doesn't mean that you get to control everything. If what you're offering doesn't stand on its own merits then what's it worth? I get that we wish Amazon couldn't do what they do but that's what the license permits.

TAForObvReasons 5y ago
The maintainers of the project chose to implement this change. You may not like it, and you are free to make a fork compatible with the original apache 2.0 license, but it is well within Elastic's right to add this. This is the danger with corporate open source in general. Facebook made a similar unilateral decision in relicensing React from Apache 2 to "BSD + Patents" many years ago. They faced quite a bit of pressure, resulting in another relicensing to MIT.
- geofft 5y ago
  No one is disputing it's within their rights. It's within their rights to add a feature to the library that blasts "Caramelldansen" through the nearest Chromecast device when you import it, too. What they're saying is that this is a breach of community norms.
- lolinder 5y ago
  They have the right to do so, but that doesn't change the fact that it's a bad idea politically. It comes off as Elastic prioritizing their war with Amazon over the community, leaving room for Amazon to continue to position themselves as the community's ally.
  - altdataseller 5y ago
    At the end of the day, CTOs and decision makers in corporations don't go with the company that position themselves as "allies". They go with the company that is shipping important new features, with better support and resiliency. If Elastic is shipping features faster than AWS is, then they'll go with Elastic. I bet most CTOs aren't even aware of the politics happening - it's only the echo chamber in Hacker News that is making a big deal of this.

MilnerRoute 5y ago

Hacker News had a good discussion about this earlier today. https://news.ycombinator.com/item?id=28103389 https://news.ycombinator.com/item?id=28103389 What surprised me was how many commenters seemed unhappy with Amazon's Open Source alternative and the level of resources they appeared to be committing to it.

thedougd 5y ago
Amazon is certainly the big bad wolf here, but I wouldn't be surprised if the move is also designed to inconvenience the small players like logz.io, Graylog, Searchly, etc who might be interfering with their flow of introductory level customers. A quick search reveals there are more of these providers than I ever realized.
marcinzm 5y ago
AWS is pretty well known for rolling out half baked products with understaffed teams especially when it's a re-hosting of an open source product. EKS was a mess for a long time and had a tiny team behind it from what I hear.

deleted 5y ago

[deleted]

iandanforth 5y ago

Good? Amazon deserves no support in branding their version 'open'.

Spivak 5y ago
You do realize that Amazon would have been happy selling hosting for the official Elastic code until Elastic forced them into a situation where the pull the rug out of all of their customers or fork? Amazon is actually the more open company when it comes to Elastic at this point. Open source does best when it’s not the meal ticket of the company developing it.
- nijave 5y ago
  It's a bit clunky, but Amazon's fork also comes with security by default. ES open-by-default has caused breaches for years. Amazon's is a bit clunky to setup but at least comes with defaults where you are less likely to leak your data

egberts 5y ago

It’s nigh time to fork the ElasticSearch API.

reilly3000 5y ago

I’m not going to say that Elastic has no real competition, but as a whole package it stands alone. It’s speed and versatility definitely differentiate it from other similar systems. The fact that it can be a log aggregator, search engine, personalization system, seim, analytics tool, and forecasting tool make it a lot more useful than typical full-text search systems, often which can’t process aggregations or run in a single instance. I think they know their position in the market is strong and that helps them feel empowered to make risky moves with their licensing. I’d rather use anything else, but for interactive analytics (sub second response times on millions of data points) it’s tough to beat. It’s a fussy app to self-host or run in a non-managed cloud environment. I wish them the best, but worry these kinds of moves will inspire more competition from totally other projects, let alone it’s fork(s).

WontonDon 5y ago
As long as they keep innovating to stay number 1, they will do just fine, even against AWS fork or new competitors.
nullify88 5y ago
A lot of people like myself just want to view logs and dont care for features like SIEM. This is where the OSS and OpenDistro versions shine. I have revisited Loki after all this new though, but I think its still missing full text search.
- chippiewill 5y ago
  > but I think its still missing full text search. That's very much by design, by not having indexing of the logging content it's much much leaner and efficient. It's aimed at a completely different use case where you already know the set of log streams you want to monitor.

teclordphrack2 5y ago

This is definitaly going to blow up in their face. I know at least a dozen people who learned open source elastic search in a research environment and then moved jobs to doing it in a for profit company.If the paid vs open source products are not going to be the same then your losing training that companies want. I also have ran into the scenario that others have described where we use the open source version for proof of concept b/4 moving to the paid version when needed. At the least, this muddies the waters and creates extra research time for engineers just to understand the licensing and cost implications of the product.

mattmcknight 5y ago

People who want open source search should take another look at Solr, it's doing fine.

mdaniel 5y ago
I don't exactly know why you're getting downvoted, but I will point out that while they play in the same sandbox, this is roughly equivalent to suggesting that someone rewrite their app because Solr and ES have absolutely stellar differences of opinion about the API used to access them, the amount of schema one must provide up-front to the document index, and the amount of operational hoops one must jump through to keep the service alive Sure, there are about 5-6 open source search engines one could pick from in a greenfield project, with amazing differences of features and maturity, but choosing between Open Search and ElasticSearch is likely a "political" decision, since they are (ahem, mostly) API compatible and have very similar operational needs.

Proven 5y ago

Why would they waste their time testing and maintaining it for the forks? Those other forks are (alsoA) open source, what's the problem? Maintain your own library if you need it.

kelnage 5y ago

Elastic, like others at the time, have used open source to their advantage - to start. The obvious one is that it is built around the Apache Lucene, and I have no doubt that this is one of the reasons ES ended up being initially released under the Apache license. Secondly, being released under a permissive open source license definitely helped with its adoption. I was working as a senior developer in a UK Government department in 2013 and we had need for a full-text search engine for a project - and even before v1, ES was a contender, and it was eventually selected once v1 was released. This was largely due to a) ease of set-up/use and b) it’s release under an open source license. If it had been under AGPL, would we have still used it? Yes, probably - our specific use case wouldn’t have been affected by such a license, and the dept. was relatively open to more complex OSS licenses - but I have worked for several other orgs. where even just the AGPL would have resulted in a hard “no”. Thirdly, ES has had contributions from a wide range of people. I honestly don’t know how I’d even begin to evaluate how much value ES has got from community contributions, but I feel it’s likely to be greater than the costs of managing those contributions. But of course eventually Elastic got funding and had shareholders to placate. I don’t really have much sympathy for them about this conflict - their early choices were in part clearly made to maximise their value, and they decided to cash in on that value at a later date - the fact that those decisions had implications for their value should have been somewhat obvious to any investor who did any due diligence. I’m still not entirely convinced that the open source model is antithetical to commercialisation, but I think it does highlight how early decisions around OSS licensing can affect such processes.

jka 5y ago
I keep hearing the AGPL-makes-corporations-wince argument, and I'm curious: what are the reasons given? Does the AGPL really place such burdens on the organization such that the benefits of a locked-open, community-guaranteed (albeit popularity not guaranteed) technology aren't worthwhile? Or is it kind of a cargo-culting and cultural-norms phenomenon where people don't use AGPL projects because they've heard that other people don't use them, thus continuing the cycle?
- kelnage 5y ago
  In one of the of the organisations I mentioned, they had a strict policy against using any GPL dependencies, let alone the AGPL. I tried discussing this with the legal policy person but they were quite resolute - they feared it’s use could “infect” our code and therefore must be avoided. I frankly doubt there’s any sort of cost-benefit analysis being done here. Certainly in my experience it was much more driven by legal uncertainty and risk-aversion.

PeterZaitsev 5y ago

Oh my. I'm not sure if there is any Open Source company which fell so low - to Break the Drivers... I'm also not sure what they are looking to reach with this step as Developers who want their applications to work both with Elastic and OpenSearch will not need to go right into AWS hands...