7 ms·
The Elasticsearch Saga Continues
- WontonDon 5y agoJust look at the the github repo activity between elastic and opensearch. Overwhelmingly, elastic is doing 100x better. AWS doesn't care as much as elastic about these products.
- nullify88 5y agoI dont think repo activity is an accurate representation since its been less than a year since Elastic made their license changes and Opensearch was shortly announced in January 2021. Give em some time to grow.
- gorgonzolachz 5y agoI would disagree here. AWS has to be strong out of the gate to push opensearch as more than just "elasticsearch, but on AWS". Elastic has proven its worth, or this fight wouldn't even be happening. AWS is simultaneously David and Goliath in this fight - they obviously won't have ES' newest, coolest features immediately, but companies will sign up with them instead of Elastic if it means they get good-enough software at a reasonable pace. Elastic's playbook is almost diametrically opposed to this: they need to prove that they're worth paying a premium for, over AWS' offering. The repo activity clearly shows that they're working on more stuff, building faster, whatever signals you want to glean from that. Change is happening, and if it's good change I can see the gambit AWS is pushing turn into a losing proposition. The endgame for Elastic is a partnership on AWS, on "equal" terms; license fees, a cut of the profits, the whole shebang. The endgame for AWS is an end to Elastic as a company, but not heavily increased profits one way or another - AWS still gets its cut if Elastic succeeds. The incentive models aren't equal here, and that's what kills AWS' agenda. I'm not affiliated with either company, so I don't have any inside info, but I have to imagine that Elastic will succeed here in carving out a niche for themselves - unless there's been a silent exodus from the company I understand that these are smart folks building software under, shall we say, extrenuous motivational circumstances. A little stress is good for productivity :)
- vladvasiliu 5y ago> Elastic's playbook is almost diametrically opposed to this: they need to prove that they're worth paying a premium for, over AWS' offering. The repo activity clearly shows that they're working on more stuff, building faster, whatever signals you want to glean from that. Change is happening, and if it's good change I can see the gambit AWS is pushing turn into a losing proposition. As an Elastic customer managing my own cluster, the main issue I have with their approach, and the main reason I'm tempted to switch to OpenSearch, is their all-or-nothing license model. We only use ES as a place to centralize logs, look them up, maybe whip up a dashboard now and then. No fancy ML or what have you. We don't (yet?) care for those. But we also need access control. There's no such thing in the free version, so we have to pay up quite a hefty sum. We could also get by with 3 data nodes. Tough. Minimum is five. We'd really like it to have some kind of SSO, like pretty much all our other apps. We'd have to pay up even more. So yeah, we'd be getting all kinds of ML-laden features and whatever. So what? Those bring 0 value to us today. So my thinking is, what if OpenSearch doesn't have the latest and greatest features? Aside from a few things around snapshot management, I was never really interested in those. Plus the license premium is so huge compared to our usage, we could just be spinning a tad bigger VMs and keep everything on-line. I wonder how many clients like us they are alienating with, what looks to me, a petty war. But as someone said on some other threads on this subject, Elastic isn't a mom-and-pop company, so our annual license costs are probably pocket change for them.
- nullify88 5y agoTheres a long list of features we don't use. We primarily use Grafana for our dashboarding needs. If I had to make a short list. Data streams, index lifecycle management and security on top of the OSS distribution of ES is all I need to view and manage K8S pod logs. Worth noting that the developers of searchguard which the Opensearch / Opendistro security plugin is based on is also being sued by Elastic.
- vladvasiliu 5y ago> Worth noting that the developers of searchguard which the Opensearch / Opendistro security plugin is based on is also being sued by Elastic. Huh, didn't know that. I really doubt their "threat" is in any way comparable to AWS. This really doesn't help whatever goodwill they had left...
- deleted 5y ago[deleted]
- easton 5y agoThis was already true of the beats as of 7.13, but still, this sucks. Time to pin some dependencies.
- taf2 5y agoLooking to upgrade a cluster running 7.5 - should I go open search or elastic? Who will be the better maintainer? Reminds me of Mariadb and MySQL… either path was a fine choice IMO but hard to say in this case…
- simlevesque 5y agoElastic.co one thousand percent. I just finished a migration from AWS managed ES to Elastic.co. You literally only need to use both for a couple of hours to notice that AWS treats ES as an afterthought. I could name many things such as better security, interface and cold/warm storage configuration.
- taf2 5y agoNot looking at either for manger service… just the open source projects- features etc…
- kyriakos 5y agoThen you have no reason not to go with elastic unless you host on aws. From the drama described in the article it sounds like elastic will try to keep clients working only on its own fork while amazon will play catch up.
- nextaccountic 5y agoBut ElasticSearch isn't open source, it's only source available.
- kyriakos 5y agohttps://github.com/elastic/elasticsearch https://github.com/elastic/elasticsearch They changed the license but its still open source
- pritambaral 5y ago
- karterk 5y agoMaking the client work only with only a specific ES version seems like an oversight[1], but it is really difficult to say given the back and forth between AWS and Elastic here. I hope it is not intentional. [1]: https://twitter.com/xeraa/status/1423071203753869313 https://twitter.com/xeraa/status/1423071203753869313
- prepend 5y ago> A few years ago, AWS basically forked ElasticSearch to offer it as a service, much to the open source community’s dismay. In response, and after some time, Elastic decided to change the licensing on ElasticSearch earlier this year to restrict its downstream use, again, much to the open source community’s dismay I get annoyed a little when people try to ascribe a feeling to a large group. I don’t think “the open source community” was dismayed about either of these actions.
- whoaisme 5y ago"I get annoyed a little when people try to ascribe a feeling to a large group. I don’t think “the open source community” was dismayed about either of these actions." Lol irony
- josephcsible 5y ago> I don’t think “the open source community” was dismayed about either of these actions. I do. A major piece of software suddenly stopped being open source.
- lacksconfidence 5y agoIt may not be open source, but it still offers much of what I think is important. Development is done in the open, very much following the bazaar model. Users of the software are free to read, modify, and release forks/patches of the source code. For the vast, vast majority of users that are not trying to sell elasticsearch as a service the terms of the license may have changed, but in practical terms all the things I care about are still there.
- kureikain 5y agoI think in long run AWS will just lost their momentum or community support and people will realize AWS won't be able to deliver. I had used AWS hosted Elasticsearch and it's subpar even at fundamental level. That signal to me they just don't care at all about search and this whole opensearch thing is just they want to be look like a good guy. If AWS doesn't even care about their own UI/UX of the dashboard or the UX of their API, how can I expect them to do a good job at Elasticsearch? They right now still ride on the high and stay close to ES, but the further into the future, when they drift from ES, expect to have a crappy API from them. In fact, looking at the Github Opensearch repo for Es and dashboard(Kibana equivalent) they are clearly less active than ES. So how can we expect them to keep the pace in next few year?
- reducesuffering 5y agoOh they care, there are just some major issues with execution. Amazon, because of their much-panned internal culture, struggles to hire the required developers for many areas. Elasticsearch... is especially bad; because that org is absolutely swamped with operating issues. It's a pressure cooker inside a pressure cooker. There are so many high-priority tickets waking up devs in the middle of the night all the time. Many reports of developers juggling 60+ high severity tickets a week. Then because of this, few people want to work there because the conditions are extra bad and the current devs leaving. It's been a mess for a long time. Not for lack of care about the business opportunity, but because management is unable to figure out a healthy way of approaching it.
- ram_rar 5y agoBy restricting their clients to only talk ES instances, they are shooting themselves in the foot. This gives AWS opensearch clients a great opportunity to enhance their clients SDKS that can support both. ES should focus on competing be with DDOG/App Dynamics and other incumbents in SIEM space. I dont understand, why would they get into petty fights with AWS.
- user5994461 5y agoIt's no small feat to maintain compatibility with other product and reverse engineer changes as they happen. Good luck with that. How many people does AWS have people working on their version of elasticsearch? and on the client libraries in various languages? From previous discussions on HN, I was under the impression that the answer is very little if any. My bet is that AWS will keep the service as it is now and never makes patches or upgrades. That's how they operate services usually. And it's not that bad to be fair, the product is good enough as it is.
- notexactlyadev 5y agoBased on their current GitHub contributions, I'd say they have quite a few folks. 50+ across opensearch, opensearch dashboards, and the various plugins if I were to guess. https://github.com/opensearch-project https://github.com/opensearch-project
- art-vandelay 5y agoYou know we're living in a society! We're supposed to follow SemVer! Regardless of whatever beef is going on between AWS and Elastic, they could at least have the professional courtesy not to ship breaking changes in a minor release.
- bradknowles 5y agoThey’re not going after the same customers. AWS is targeting people who are already using AWS and who want to use an ES type service, but without managing their own servers running the ES software. They want something they can plug into their CloudFormation templates and let AWS do all the low-level admin work. ES the company is going after people who want all the latest bells and whistles and geegaws, and are willing to build and maintain their own ES clusters to make that happen. Yes, there’s some overlap in the middle, but not much. And no matter how bad the ES service is on AWS, it will always have a large customer base who will choose it, simply because it’s good enough, as well as easier and lower administrative overhead. The OpenSearch thing is new. So far as I know, they didn’t create their own forks of MySQL or Postgres and then try to build communities around that. But that is what they’re doing with OpenSearch. IMO, that’s a mistake. They’re better off working with the ES community and contributing changes back to an existing community, as opposed to trying to build and maintain their own separate but equal community. Sometimes forks are a good idea. Sometimes not. Sometimes those forks live on, and sometimes not. This is a case where I think the fork is a bad idea, I believe it is likely to not live on, and I think a lot of harm will be done on both sides of that war, above and beyond whatever else might have happened if they had just chosen to support the existing ES community. But that’s just my personal opinion.
- alichapman 5y agoAmazom were contributing changes back to Elasticsearch before the license change - the license change basically explicitly says "you can't run this as a service and sell it to people". With this in mind what could Amazon do? Their options were: A. Stop selling their Elasticsearch service, or B. Fork Elasticsearch. This change from Elastic is really shitty though, and sort of proves that all their finger pointing at Amazon saying that they're acting against the spirit of open source was all bullshit.
- martypitt 5y agoThose aren't the only options. AWS could've paid to license the software.
- 5y ago
- catmanjan 5y agoCan't Amazon just serve these bogus headers and keep opensearch compatible? It's not like headers are copyrighted...
- mcintyre1994 5y agoI guess at an extreme Elastic could do some public/private cryptography with the open source client using a public key to verify headers encrypted per request with a private key kept in their cloud service. It sounds ridiculous for an open source project to do that, but it must be completely controlled by Elastic for them to have made the change they already have. So ultimately they probably do need to move to new clients that Elastic don’t control even if they can do that for now.
- doteka 5y agoHuh, pretty incredible how Elastic took a classic David vs Goliath scenario and managed to get people rooting for Goliath.
- nix23 5y agoCryengine did exactly the same...
- nextaccountic 5y agoCan you share this story?
- nix23 5y agoSure, so StarCitizen (the game) used Cryengine at first but Crytek (the company behind Cryengine) was not doing well, and some talents left it. In the meantime Amazon took Cryengine over and renamed it to Lumberyard (now Open 3D Engine). StarCitizen then turned to Lumberyard because..well the because the state of Crytek, and Crytek sued SC, it a bit more complicated why CT sued SC sure, but no one want to work with Crytek nowadays because of that. https://www.polygon.com/2020/2/21/21147664/crytek-star-citizen-lawsuit-cloud-imperium-games-roberts-space-industries-cryengine-squadron-42 https://www.polygon.com/2020/2/21/21147664/crytek-star-citiz...