4 ms·
And I, for sure, will never use them. The AWS version of ES has been abysmal- it’s only saving grace is that it’s “in the ecosystem”- I was convinced by an AWS
by dijit 6y ago
And I, for sure, will never use them.
The AWS version of ES has been abysmal- it’s only saving grace is that it’s “in the ecosystem”- I was convinced by an AWS zealot on my team. Never again.
- uncledave 6y agoI’ll second that. Complete pile of excrement. Edit: See my comment further down for an extrapolation. This one lacks merit which was a bad call on my part.
- afandian 6y agoWhat exactly is wrong? And did you switch to something else?
- uncledave 6y agoOutages during scale out, weird insurmountable restrictions on cross region replication, cost, backup and restore doesn’t work properly, cluster irrecoverable corruption and it is 100% opaque whenever something goes wrong. This results in escalation to AWS support who barely understand it. The big sell was easy deployment and cost management but the negatives outweighed that. We tried Elastic themselves but the sales and technical process was a nightmare. It was virtually impossible to work out what we were going to have to spend. We needed X-Pack (RBAC and SSO) but the enterprise licensing is all over the place for it. Ergo we didn’t bother and stuck with our DBMS FTE stuff. It’s crap but the ROI is better.
- ArchOversight 6y agoPart of the issue is lack of visibility into what is going on, you don't get a lot of ability to debug. Company I was at switched from AWS's ElasticSearch to hosting it ourselves on top of EC2 instances so that we could control the memory/CPU/disks and see logging/understand more about the internals to help understand why it was slow for our use case.
- core-questions 6y agoI send about 100GB a day to Amazon ES and it works fine. I used to maintain ES 2.x and 5.x on my own and it was more work for me personally at a slight cost savings. What has been abysmal for you? Maybe your use case is more advanced than ours, which is mainly absorbing logs from all over the place and doing the typical dashboard and alerts on them (with Grafana).
- halbritt 6y ago> I send about 100GB a day to Amazon ES This is why you haven't noticed any issues.
- tinco 6y agoAre you insinuating that's a small amount of data? You've got to be kidding.
- mcintyre1994 6y agoIt is, relative to the scale AWS ES is apparently built to support. > Amazon Elasticsearch Service lets you store up to 3 PB of data in a single cluster, enabling you to run large log analytics workloads via a single Kibana interface. https://aws.amazon.com/elasticsearch-service/ https://aws.amazon.com/elasticsearch-service/
- fastball 6y agoThat's the absolute maximum case though, literally. "You're not the DoD so of course you're not having issues". It can't be more than a minuscule fraction of Amazon ES customers that are getting anywhere close to 3PB.
- geerlingguy 6y agoI've seen devs toss crud into infra with debug logs enabled, with millions of lines of deprecated log messages, etc., and the infra budget eats their costs. It's insane. Unless you're literally Facebook, or ingesting data from CERN's LHC... what possible use case requires 100GB of text data ingest per day? Maybe it's a case of someone throwing a Service Mesh into a Microservices K8s cluster and logging all the things?
- mrsuprawsm 6y agoPersonally me and my team just evaluated the AWS ES and Elastic offerings, and the AWS offering (surprisingly!) came out on top, for our use case. Better performance, better IaC support, and marginally cheaper. Honestly I would have preferred the Elastic offering to work better, but that wasn't the case.
- deleted 6y ago[deleted]
- dawnerd 6y agoWe used to use ES, but after multiple issues with the index getting corrupt for various reasons we decided to actually save money and use Algolia (before their price structure change - now it would be more expensive). Over the last year+ we've had no issues with search or indexing.
- inssein 6y agoMight want to elaborate a bit. It is true that from a feature point of view, Elastic's own offering wins out, but from a perspective of uptime, cost, and performance, AWS wins.
- dijit 6y agoI guess it depends on who’s managing your elk. I’ve been running elks stacks for just about 8 years in some form or another, AWS’ was very poor in availability compared to what it should have been. There was a bunch of issues honestly, large messages got truncated, huge stalls in throughput (like 300-500ms of freezing), the fact that it was missing a bunch of features and was slower to query was just icing on the cake.