3 ms·

Elasticsearch recommends <32GB because of 32-bit per their doc, is it different for coordinator nodes?

by CSDude 6y ago

adamtulinius 6y ago
That recommendation is only for the JVM heap to my knowledge.
- Xylakant 6y ago
  For coordinator nodes, only the HEAP is relevant, excess RAM for disk buffers is only of interest for data holding nodes.
Xylakant 6y ago
The underlying reason is that at around 32GB the JVM internally shifts from 32bit pointers for HEAP management to 64bit. The feature is called compressed object pointers (compressed oops) The increased pointer size uses substantial amounts of RAM, a reasonable estimate is that you need at least ~45GB to not have a loss in total available HEAP. Also, GC cycles get longer the more HEAP you have. However, some workloads may require more heap to even be possible and there’s pauseless garbage collector implementations able to handle substantial amounts of memory, so it is possible. It’s just not something you should be doing unless you know exactly what you are doing. See https://wiki.openjdk.java.net/display/HotSpot/CompressedOops https://wiki.openjdk.java.net/display/HotSpot/CompressedOops
- reitzensteinm 6y ago
  Lacking value types, larger pointers will probably penalize JVM programs more than their peers.
  - SomewhatLikely 6y ago
    While there are no JEP 169 style Value Types, you can still layout your data in a way that significantly reduces the number of pointers. For example using parallel arrays of primitives. It's not as clean, but it is one mitigation.
behohippy 6y ago
A "perfect" ES node has 64 gig of memory. 32 to heap and 32 for off heap as well as 8-12 cores for executing queries. Heap is mostly used for internal operations and the inverted index stuff (fielddata), off-heap was used for the columnar data store (docvalues). Most server class machines exceed this nowadays, so we would recommend containerizing multiple instances of ES on a single machine and using the rack awareness feature to make sure shards didn't replicate onto the same physical host. It's been a while since I worked at Elastic, so this info might be out of date by now, but we're still using this formula for our ECE setups where I work now.

Elasticsearch recommends <32GB because of 32-bit per their doc, is it different for coordinator nodes?

by CSDude 6y ago

Elasticsearch recommends <32GB because of 32-bit per their doc, is it different for coordinator nodes?

adamtulinius 6y ago

That recommendation is only for the JVM heap to my knowledge.

Xylakant 6y ago
For coordinator nodes, only the HEAP is relevant, excess RAM for disk buffers is only of interest for data holding nodes.

Xylakant 6y ago

The underlying reason is that at around 32GB the JVM internally shifts from 32bit pointers for HEAP management to 64bit. The feature is called compressed object pointers (compressed oops) The increased pointer size uses substantial amounts of RAM, a reasonable estimate is that you need at least ~45GB to not have a loss in total available HEAP. Also, GC cycles get longer the more HEAP you have. However, some workloads may require more heap to even be possible and there’s pauseless garbage collector implementations able to handle substantial amounts of memory, so it is possible. It’s just not something you should be doing unless you know exactly what you are doing. See https://wiki.openjdk.java.net/display/HotSpot/CompressedOops https://wiki.openjdk.java.net/display/HotSpot/CompressedOops

reitzensteinm 6y ago
Lacking value types, larger pointers will probably penalize JVM programs more than their peers.
- SomewhatLikely 6y ago
  While there are no JEP 169 style Value Types, you can still layout your data in a way that significantly reduces the number of pointers. For example using parallel arrays of primitives. It's not as clean, but it is one mitigation.

behohippy 6y ago

A "perfect" ES node has 64 gig of memory. 32 to heap and 32 for off heap as well as 8-12 cores for executing queries. Heap is mostly used for internal operations and the inverted index stuff (fielddata), off-heap was used for the columnar data store (docvalues). Most server class machines exceed this nowadays, so we would recommend containerizing multiple instances of ES on a single machine and using the rack awareness feature to make sure shards didn't replicate onto the same physical host. It's been a while since I worked at Elastic, so this info might be out of date by now, but we're still using this formula for our ECE setups where I work now.