7 ms·

what is preventing AWS from dropping OpenSearch and going back to just selling Elastic ?

antimemetics 2y ago
sunk cost. at least that’s what elastic seems to be betting on.
mr90210 2y ago
Perhaps the fact that AWS has got years of time investment already made into OpenSearch and staying with their own version allows them to reduce risks should Elastic attempt to change ElasticSearch’s license again.
foxyv 2y ago
To be honest, the OpenSearch brand has more value now than Elastic.
- jdboyd 2y ago
  Please explain. This isn't obvious to me.
- lolinder 2y ago
  I see a couple people on here claiming this, but no data to back it up. Elastic beats out OpenSearch by a wide margin on every metric I've thought to check (gh stars, gh stars rate of increase, number of commits, number of pull requests opened, number of pull requests merged, number of issues, stack overflow questions...). Not a single one shows OpenSearch ahead. What metric are you using to come to the conclusion that OpenSearch is the more valuable brand?
  - jdc0589 2y ago
    thats a super one sided set of metrics. it doesn't tell us anything about how many people are actually using one, just how much visible dev activity they have. I don't have those metrics or an opinion, im just saying that value is based on utilization by a product's target users, not support activities.
    - lolinder 2y ago
      > it doesn't tell us anything about how many people are actually using one, just how much visible dev activity they have Stars and rate of star growth and stack overflow activity are all passable proxies. They're not great, and I'm open to better metrics, but they're what I can find. Truly, if anyone can give me any metric that shows OpenSearch ahead I'll shut up. I can't find one, and I've looked.
      - grogenaut 2y ago
        AWS revenue. Internal AWS stats. Open search client pulls (not sure if the client is different now). Opensearch is mainly going to be used on AWS so that won't show up as much in GitHub stars.
        lolinder 2y ago
        None of those metrics measure the value of the brand, so none of those metrics are useful for the question that this thread is about: figuring out if AWS benefits from sticking with the OpenSearch brand. If I'm right that OpenSearch has a weak brand, Amazon could switch and their internal stats and revenue wouldn't budge.
  - PeterCorless 2y ago
    As a related proxy for market awareness, DB-Engines.com's ranking is a composite index of a number of factors: • Google search volume and quantity of search result pages (blogs, recipes, etc.) • Mentions in LinkedIn profiles as a skill • Number of job post listings as a requisite skill • Social media mentions It is a measure of "mindshare" or "share of voice." Not one of market share ($$$) or utilization (TBs under management, etc.). With that said, in the August 2024 listing: • Elasticsearch is ranked #8 (of 423 systems tracked), with an index score of 129.83 • OpenSearch is ranked #35, with an index score of 16.47 This would make OpenSearch about an eighth as prevalent.
    - lmeyerov 2y ago
      Relevant, the curves tell a different story: https://db-engines.com/en/ranking_trend/system/Elasticsearch%3BOpenSearch%3BSplunk https://db-engines.com/en/ranking_trend/system/Elasticsearch... A big question is New + Churning installs. I'd expect a scary portion of the New, and inherently slower rate of Churn are heading to OS instead ES. The curves support that: note that ES isn't substantively growing on that chart. Anecdotally, we see most new installs as leaning to OS in the security industry, which is one of the top money makers for ES/OS.
      - PeterCorless 2y ago
        Yes, exactly. I often do comparative results like this to see where there is "momentum." Opensearch is definitely coming on strong.
  - cqqxo4zV46cp 2y ago
    [dead]
  - pheatherlite 2y ago
    For one, we don't have to pay for basic amenities like security and alerts. To heck with gouging the customer for basic feature sets. Aws have their faults, but enabling teams to get the whole elastic experience without the weird nickle and diming is a blessing. Good on Amazon and boo elastic.
    - lolinder 2y ago
      This is a reason why you prefer it but not evidence that the brand has overtaken Elastic's.
      - hluska 2y ago
        This evidence can only come from financial accounting. Amazon does not report OpenSearch results separately so you’re looking for data that does not exist in a public form. It’s a waste of time.
        lolinder 2y ago
        > This evidence can only come from financial accounting. That's not true—we're talking about brand value, not financial value of the product. If AWS switched over to offering ElasticSearch again (not that they will) and ditched OpenSearch, I have no reason to believe that their financial numbers would go down a bit. Brand value is nearly impossible to measure, but to the extent that you can it'd be by measuring perception among those outside the company, not through an accounting of the company's actual revenue. Think of it this way: the brand LENRUE [0] is worth approximately zero. The company that makes these products could rebrand tomorrow and their revenue stream wouldn't take a hit in the slightest. But the company presumably actually makes some amount of money. For Elastic vs OpenSearch, the brand value of the two products should be loosely comparable by looking at some measures of public perceptions, and I can't find any measurement that would suggest OpenSearch is in the lead. [0] https://www.amazon.com/stores/LENRUE/page/24E9713E-AC7E-4269-81CF-496AF4ABCE91 https://www.amazon.com/stores/LENRUE/page/24E9713E-AC7E-4269...
  - lmeyerov 2y ago
    Louie.ai team talks to a lot of sec teams, and new projects generally prefer OS bc openness, and a lot of migrations. Think MSPs etc. Anecdata, but very real.
  - cdelsolar 2y ago
    interesting. I've heard of ElasticSearch a gazillion times and this is the first time I've heard of "OpenSearch", and I've also been using AWS since it came out basically.
  - foxyv 2y ago
    I'm basing it off my own experience with managers, developers, and sysadmins. No one wants Elastic Search over Open Search. We're spending all our money on Open Search. Whenever I mention Elastic Search I may as well have just farted. Everyone hates the support and enterprise licensing model. We already pay for AWS support anyways so why not use their fork? Why add yet another vendor to our tech stack?
- gip 2y ago
  I think it's nuanced. I'm currently managing an engineering team on an interim basis. During standup I noticed that most engineers were talking about 'elastic' or 'elasticsearch' but one was talking about opensearch. I asked them to clarify that for me and they told me that they transitioned to AWS opensearch but still use the old name for the product often - that might point to the strength of the Elastic brand.
  - dblock 2y ago
    "Elasticsearch" seems to be often referred to as a technology, as in ELK stack, rather than as in "Elastic" the company.
beachy 2y ago
Large tech companies care about their reputation. If they dropped their own fork and went back and started again from Elastic that would be admitting their own incompetence. So not happening.
- 0cf8612b2e1e 2y ago
  I think the bigger risk to Amazon: what if Elastic wants to pull the rug out from under them once again? Why take that risk when Amazon is already stable? Given the duration since the fork, I suspect there are more than a couple of features differences between the products that would have to be smoothed over.
LamaOfRuin 2y ago
It's a very different open source license than their previous one. AGPL vs Apache 2.0
wmf 2y ago
If you try to sell AGPL software the owner will probably shake you down.
- davidgerard 2y ago
  Examples?
  - paulmd 2y ago
    lowagie/itextpdf being an obvious/prominent one. it happens - why would you think commercial operators who’ve chosen a dual-license model wouldn’t protect their IP? That’s literally their breadwinner.
    - davidgerard 2y ago
      "shakedown" is a rather stronger claim than "adopted a licence".
      - paulmd 2y ago
        I fairly well knew that your question wasn't good-faith, that's why I hesitated to respond rather than just flagging it.
        davidgerard 2y ago
        what element was "bad faith", that the query was nonsense? Point me at anything that would legally or even ethically constitute a "shakedown" as alleged by OP, who it appears in the subthread to have misread the license? like, answer the question instead of evading, surely it cannot be that hard.
  - wmf 2y ago
    Grafana https://grafana.com/blog/2021/04/20/qa-with-our-ceo-on-relicensing/ https://grafana.com/blog/2021/04/20/qa-with-our-ceo-on-relic... RethinkDB http://web.archive.org/web/20161003231505/https://rethinkdb.com/faq/ http://web.archive.org/web/20161003231505/https://rethinkdb.... BerkeleyDB http://web.archive.org/web/20170211004058/http://www.infoworld.com/article/2611450/open-source-software/oracle-switches-berkeley-db-license.html http://web.archive.org/web/20170211004058/http://www.infowor... MongoDB tried the shakedown but I guess it didn't work http://web.archive.org/web/20240202161503/https://www.mongodb.com/legal/licensing/server-side-public-license/faq http://web.archive.org/web/20240202161503/https://www.mongod...
godber 2y ago
OpenSearch really appears to have a significant amount of development momentum. It is functionally equivalent with Elasticsearch. There would be no incentive for AWS to do this. After ES broke their compatibility with OpenSearch they also broke compatibility with their own older versions. This forced users to choose in order to move forward with anything. I don’t know how it worked out for others but I’ve converted hundreds of ES clusters to OS and switched all development to OpenSearch because they started gaining momentum and didn’t make decisions that were actively hostile to their users. Edit: As long as OpenSearch doesn’t start breaking the self hosted use case, I don’t see any reason to consider Elasticsearch again. In fact, ES would have to offer up a significant advantage to overcome the bother.
deleted 2y ago
[deleted]
CuriousCosmic 2y ago
well importantly, Elastic was originally Apache-2.0. Then Amazon started using their stuff and they relicensed to their not-really-FOSS-license. Then Amazon forked their stuff, a bunch of legal conflicts happened. Now they are friends again. And now they relicensed to AGPL (which is FOSS). So they were originally Apache-2.0 which was permissive versus now they are AGPL which is copyleft. The important distinction here is that if Amazon was to use Elastic directly, they'd have to make their contributions available to users and those users could then upstream those contributions back to Elastic. In the old situation with Apache-2.0, Amazon could take contributions from Elastic but then they kept them themself for the most part without up-streaming. This forces a give-and-take relationship vs a one-way relationship. Also importantly now that Elastic is AGPL they can integrate anything they want from OpenSearch's Apache-2.0 licensed projects but unless OpenSearch becomes AGPL as well, they can't pull any contributions from Elastic.
- wokwokwok 2y ago
  Realllllly? I sure hope not. There going to be some hard irony here, of making such a fuss about it before, when it was someone else doing it, and then doing it themselves. We’ll see. Maybe they’re principled enough not to rip off the open search contributions. If not, you’ve really got to believe there’s no sincerity left at elastic. I guess time will tell; I’d like to believe they’re better than that.
  - CuriousCosmic 2y ago
    There's a difference though. Prior to the relicensing, Amazon had a private fork they were running and weren't contributing upstream. Only after the relicense did they open source their fork. Now you have two open source projects and one has a copyleft license and the other has a permissive license. Taking contributions from the permissive to the copyleft project isn't ripping off contributions. It's using open source software and collaborating in the FOSS ecosystem. And Amazon would be free to pull contributions back the other way just as well as long as they agree to the mutual terms of the AGPL (which is by all means a FOSS license).
luceneuser 2y ago
It will not happen. OpenSearch grows fast in the last four years. There is no incentive to go back to ES.
h1fra 2y ago
Recently tried OpenSearch, it has good momentum but tbf the tooling / documentation and support are not that great. Contrary to what other people are saying in this thread, I would not say it has better branding than ElasticSearch and that ES has lost its battle. Outside AWS OpenSearch is still not a big contender
- dblock 2y ago
  You're right about documentation and tooling. There's some good momentum in the OpenSearch community around it though, and I am really excited by things like https://github.com/opensearch-project/documentation-website/issues/7700 https://github.com/opensearch-project/documentation-website/....

sunk cost. at least that’s what elastic seems to be betting on.

Perhaps the fact that AWS has got years of time investment already made into OpenSearch and staying with their own version allows them to reduce risks should Elastic attempt to change ElasticSearch’s license again.

foxyv 2y ago

To be honest, the OpenSearch brand has more value now than Elastic.

jdboyd 2y ago
Please explain. This isn't obvious to me.
lolinder 2y ago
I see a couple people on here claiming this, but no data to back it up. Elastic beats out OpenSearch by a wide margin on every metric I've thought to check (gh stars, gh stars rate of increase, number of commits, number of pull requests opened, number of pull requests merged, number of issues, stack overflow questions...). Not a single one shows OpenSearch ahead. What metric are you using to come to the conclusion that OpenSearch is the more valuable brand?
- jdc0589 2y ago
  thats a super one sided set of metrics. it doesn't tell us anything about how many people are actually using one, just how much visible dev activity they have. I don't have those metrics or an opinion, im just saying that value is based on utilization by a product's target users, not support activities.
  - lolinder 2y ago
    > it doesn't tell us anything about how many people are actually using one, just how much visible dev activity they have Stars and rate of star growth and stack overflow activity are all passable proxies. They're not great, and I'm open to better metrics, but they're what I can find. Truly, if anyone can give me any metric that shows OpenSearch ahead I'll shut up. I can't find one, and I've looked.
    - grogenaut 2y ago
      AWS revenue. Internal AWS stats. Open search client pulls (not sure if the client is different now). Opensearch is mainly going to be used on AWS so that won't show up as much in GitHub stars.
      - lolinder 2y ago
        None of those metrics measure the value of the brand, so none of those metrics are useful for the question that this thread is about: figuring out if AWS benefits from sticking with the OpenSearch brand. If I'm right that OpenSearch has a weak brand, Amazon could switch and their internal stats and revenue wouldn't budge.
- PeterCorless 2y ago
  As a related proxy for market awareness, DB-Engines.com's ranking is a composite index of a number of factors: • Google search volume and quantity of search result pages (blogs, recipes, etc.) • Mentions in LinkedIn profiles as a skill • Number of job post listings as a requisite skill • Social media mentions It is a measure of "mindshare" or "share of voice." Not one of market share ($$$) or utilization (TBs under management, etc.). With that said, in the August 2024 listing: • Elasticsearch is ranked #8 (of 423 systems tracked), with an index score of 129.83 • OpenSearch is ranked #35, with an index score of 16.47 This would make OpenSearch about an eighth as prevalent.
  - lmeyerov 2y ago
    Relevant, the curves tell a different story: https://db-engines.com/en/ranking_trend/system/Elasticsearch%3BOpenSearch%3BSplunk https://db-engines.com/en/ranking_trend/system/Elasticsearch... A big question is New + Churning installs. I'd expect a scary portion of the New, and inherently slower rate of Churn are heading to OS instead ES. The curves support that: note that ES isn't substantively growing on that chart. Anecdotally, we see most new installs as leaning to OS in the security industry, which is one of the top money makers for ES/OS.
    - PeterCorless 2y ago
      Yes, exactly. I often do comparative results like this to see where there is "momentum." Opensearch is definitely coming on strong.
- cqqxo4zV46cp 2y ago
  [dead]
- pheatherlite 2y ago
  For one, we don't have to pay for basic amenities like security and alerts. To heck with gouging the customer for basic feature sets. Aws have their faults, but enabling teams to get the whole elastic experience without the weird nickle and diming is a blessing. Good on Amazon and boo elastic.
  - lolinder 2y ago
    This is a reason why you prefer it but not evidence that the brand has overtaken Elastic's.
    - hluska 2y ago
      This evidence can only come from financial accounting. Amazon does not report OpenSearch results separately so you’re looking for data that does not exist in a public form. It’s a waste of time.
      - lolinder 2y ago
        > This evidence can only come from financial accounting. That's not true—we're talking about brand value, not financial value of the product. If AWS switched over to offering ElasticSearch again (not that they will) and ditched OpenSearch, I have no reason to believe that their financial numbers would go down a bit. Brand value is nearly impossible to measure, but to the extent that you can it'd be by measuring perception among those outside the company, not through an accounting of the company's actual revenue. Think of it this way: the brand LENRUE [0] is worth approximately zero. The company that makes these products could rebrand tomorrow and their revenue stream wouldn't take a hit in the slightest. But the company presumably actually makes some amount of money. For Elastic vs OpenSearch, the brand value of the two products should be loosely comparable by looking at some measures of public perceptions, and I can't find any measurement that would suggest OpenSearch is in the lead. [0] https://www.amazon.com/stores/LENRUE/page/24E9713E-AC7E-4269-81CF-496AF4ABCE91 https://www.amazon.com/stores/LENRUE/page/24E9713E-AC7E-4269...
- lmeyerov 2y ago
  Louie.ai team talks to a lot of sec teams, and new projects generally prefer OS bc openness, and a lot of migrations. Think MSPs etc. Anecdata, but very real.
- cdelsolar 2y ago
  interesting. I've heard of ElasticSearch a gazillion times and this is the first time I've heard of "OpenSearch", and I've also been using AWS since it came out basically.
- foxyv 2y ago
  I'm basing it off my own experience with managers, developers, and sysadmins. No one wants Elastic Search over Open Search. We're spending all our money on Open Search. Whenever I mention Elastic Search I may as well have just farted. Everyone hates the support and enterprise licensing model. We already pay for AWS support anyways so why not use their fork? Why add yet another vendor to our tech stack?
gip 2y ago
I think it's nuanced. I'm currently managing an engineering team on an interim basis. During standup I noticed that most engineers were talking about 'elastic' or 'elasticsearch' but one was talking about opensearch. I asked them to clarify that for me and they told me that they transitioned to AWS opensearch but still use the old name for the product often - that might point to the strength of the Elastic brand.
- dblock 2y ago
  "Elasticsearch" seems to be often referred to as a technology, as in ELK stack, rather than as in "Elastic" the company.

beachy 2y ago

Large tech companies care about their reputation. If they dropped their own fork and went back and started again from Elastic that would be admitting their own incompetence. So not happening.

0cf8612b2e1e 2y ago
I think the bigger risk to Amazon: what if Elastic wants to pull the rug out from under them once again? Why take that risk when Amazon is already stable? Given the duration since the fork, I suspect there are more than a couple of features differences between the products that would have to be smoothed over.

LamaOfRuin 2y ago

It's a very different open source license than their previous one. AGPL vs Apache 2.0

wmf 2y ago

If you try to sell AGPL software the owner will probably shake you down.

davidgerard 2y ago
Examples?
- paulmd 2y ago
  lowagie/itextpdf being an obvious/prominent one. it happens - why would you think commercial operators who’ve chosen a dual-license model wouldn’t protect their IP? That’s literally their breadwinner.
  - davidgerard 2y ago
    "shakedown" is a rather stronger claim than "adopted a licence".
    - paulmd 2y ago
      I fairly well knew that your question wasn't good-faith, that's why I hesitated to respond rather than just flagging it.
      - davidgerard 2y ago
        what element was "bad faith", that the query was nonsense? Point me at anything that would legally or even ethically constitute a "shakedown" as alleged by OP, who it appears in the subthread to have misread the license? like, answer the question instead of evading, surely it cannot be that hard.
- wmf 2y ago
  Grafana https://grafana.com/blog/2021/04/20/qa-with-our-ceo-on-relicensing/ https://grafana.com/blog/2021/04/20/qa-with-our-ceo-on-relic... RethinkDB http://web.archive.org/web/20161003231505/https://rethinkdb.com/faq/ http://web.archive.org/web/20161003231505/https://rethinkdb.... BerkeleyDB http://web.archive.org/web/20170211004058/http://www.infoworld.com/article/2611450/open-source-software/oracle-switches-berkeley-db-license.html http://web.archive.org/web/20170211004058/http://www.infowor... MongoDB tried the shakedown but I guess it didn't work http://web.archive.org/web/20240202161503/https://www.mongodb.com/legal/licensing/server-side-public-license/faq http://web.archive.org/web/20240202161503/https://www.mongod...

godber 2y ago

OpenSearch really appears to have a significant amount of development momentum. It is functionally equivalent with Elasticsearch. There would be no incentive for AWS to do this. After ES broke their compatibility with OpenSearch they also broke compatibility with their own older versions. This forced users to choose in order to move forward with anything. I don’t know how it worked out for others but I’ve converted hundreds of ES clusters to OS and switched all development to OpenSearch because they started gaining momentum and didn’t make decisions that were actively hostile to their users. Edit: As long as OpenSearch doesn’t start breaking the self hosted use case, I don’t see any reason to consider Elasticsearch again. In fact, ES would have to offer up a significant advantage to overcome the bother.

deleted 2y ago

[deleted]

CuriousCosmic 2y ago

well importantly, Elastic was originally Apache-2.0. Then Amazon started using their stuff and they relicensed to their not-really-FOSS-license. Then Amazon forked their stuff, a bunch of legal conflicts happened. Now they are friends again. And now they relicensed to AGPL (which is FOSS). So they were originally Apache-2.0 which was permissive versus now they are AGPL which is copyleft. The important distinction here is that if Amazon was to use Elastic directly, they'd have to make their contributions available to users and those users could then upstream those contributions back to Elastic. In the old situation with Apache-2.0, Amazon could take contributions from Elastic but then they kept them themself for the most part without up-streaming. This forces a give-and-take relationship vs a one-way relationship. Also importantly now that Elastic is AGPL they can integrate anything they want from OpenSearch's Apache-2.0 licensed projects but unless OpenSearch becomes AGPL as well, they can't pull any contributions from Elastic.

wokwokwok 2y ago
Realllllly? I sure hope not. There going to be some hard irony here, of making such a fuss about it before, when it was someone else doing it, and then doing it themselves. We’ll see. Maybe they’re principled enough not to rip off the open search contributions. If not, you’ve really got to believe there’s no sincerity left at elastic. I guess time will tell; I’d like to believe they’re better than that.
- CuriousCosmic 2y ago
  There's a difference though. Prior to the relicensing, Amazon had a private fork they were running and weren't contributing upstream. Only after the relicense did they open source their fork. Now you have two open source projects and one has a copyleft license and the other has a permissive license. Taking contributions from the permissive to the copyleft project isn't ripping off contributions. It's using open source software and collaborating in the FOSS ecosystem. And Amazon would be free to pull contributions back the other way just as well as long as they agree to the mutual terms of the AGPL (which is by all means a FOSS license).

luceneuser 2y ago

It will not happen. OpenSearch grows fast in the last four years. There is no incentive to go back to ES.

h1fra 2y ago

Recently tried OpenSearch, it has good momentum but tbf the tooling / documentation and support are not that great. Contrary to what other people are saying in this thread, I would not say it has better branding than ElasticSearch and that ES has lost its battle. Outside AWS OpenSearch is still not a big contender

dblock 2y ago
You're right about documentation and tooling. There's some good momentum in the OpenSearch community around it though, and I am really excited by things like https://github.com/opensearch-project/documentation-website/issues/7700 https://github.com/opensearch-project/documentation-website/....