3 ms·

Thanks mate. I did tried Elastic before settling on AWS ES, it was slower somehow. As for Kibana features, im happy with barebones :) > you use e.g. file or do

by nuker 5y ago

Thanks mate. I did tried Elastic before settling on AWS ES, it was slower somehow. As for Kibana features, im happy with barebones :) > you use e.g. file or docker beats for collecting logs. My setup is custom app reading from CW Logs. But can Elastic cluster scale automatically on indexing latency spikes, so my apps writes do not time out? If yes then how, please?

jillesvangurp 5y ago
Autoscaling is of course something they can do: https://www.elastic.co/guide/en/cloud/current/ec-autoscaling.html https://www.elastic.co/guide/en/cloud/current/ec-autoscaling... If you use life cycle management and data streams (which AWS doesn't have), you'd be able to control the sizes of your hot indices (i.e. the ones you write to). Basically keeping your hot indices small helps keeping things fast. If you have issues with app writes spiking, I'd use some queuing solution in between. Plenty of solutions for that. The rest is just a matter of configuring things right in terms of number of shards and setting up properly. Basically, you get what you pay for in the end.
- nuker 5y ago
  > If you use life cycle management and data streams (which AWS doesn't have), you'd be able to control the sizes of your hot indices (i.e. the ones you write to) AWS has lifecycle stuff, a year ago. And index management is not cluster scaling, its a good advice, but not what I asked about.
- nuker 5y ago
  To add. Your link to autoscaling describes storage scaling up, not the cluster scaling based on performance metrics.