3 ms·
Just upgraded an AWS cluster from Elasticsearch 5.5 to Opensearch 1.3 - which is just really is an ES 7.10 fork. Those are 4 years apart and many releases in be
by flockonus 4y ago
Just upgraded an AWS cluster from Elasticsearch 5.5 to Opensearch 1.3 - which is just really is an ES 7.10 fork. Those are 4 years apart and many releases in between, I was expecting some expressive cost reduction, but not at all, it took the exact amount of VMs for the cluster to stay above water.
- xeraa 4y agoElasticsearch in later versions has done quite a lot in terms of memory reduction. Last section of https://www.elastic.co/blog/three-ways-improved-elasticsearch-scalability https://www.elastic.co/blog/three-ways-improved-elasticsearc... but it has been an ongoing effort in multiple of the later 7.x and and 8.x releases.
- nullify88 4y agoI would have expected some memory savings due to the indexes being lighter on the heap. Perhaps your workloads are more cpu heavy than memory?
- flockonus 4y agoQuite likely yes. I can't share the graph, but memory pressure didn't alleviate at all as I'd hope it would. We'll be experimenting with different VM types next.
- karlney 4y agoShameless plug, but one of the other blog posts in the above series explain how we addressed high and unpredictable memory load in relation to wildcard searches. https://underthehood.meltwater.com/blog/2022/11/25/how-we-upgraded-an-old-3pb-large-elasticsearch-cluster-without-downtime-part-3-search-performance-and-wildcards/ https://underthehood.meltwater.com/blog/2022/11/25/how-we-up... After those changes were made then we haven't had any major issues with heap spikes. But it depends on a lot of factors, how your data looks, how your queries look and what aggregations that are made so its very hard to give some always applicable advice.