4 ms·
Meanwhile, my biggest issue with elasticsearch is that for any kind of system where you have indices of different sizes (eg you have several data models being i
by vasco 9y ago
Meanwhile, my biggest issue with elasticsearch is that for any kind of system where you have indices of different sizes (eg you have several data models being indexed), your cluster will make no attempt at trying to distribute smartly and instead will stop at "same number of indices per node". Which means you very often end up with a very unbalanced cluster with regards to disk usage. For earlier versions there was a plugin that had a smarter allocator, which would look at disk usage when assigning nodes, but for any recent ES cluster (>2.0) you're left with a shrug.
- WingH 9y agoI have this issue as well. I just over-allocate nodes to my cluster so every node is comfortably below a disk usage threshold rather than waste time on optimizing this.
- deleted 9y ago[deleted]
- itruslove 9y agoWould changing the disk high water mark setting help? https://www.elastic.co/guide/en/elasticsearch/reference/current/disk-allocator.html https://www.elastic.co/guide/en/elasticsearch/reference/curr...
- andrewvc 9y agoHave you seen https://www.elastic.co/guide/en/elasticsearch/reference/current/disk-allocator.html#disk-allocator https://www.elastic.co/guide/en/elasticsearch/reference/curr... ? Elasticsearch does take into account disk usage when balancing shards. Perhaps you haven't yet hit the (configurable) high water mark where that rebalancing would occur? I work for Elastic, but not on Elasticsearch.