4 ms·
We achieved more than 3 times speedup using "on-demand" transparent huge pages in ClickHouse[1] for a very narrow use-case: random access to a hash table that d
by zX41ZdbW 4y ago
We achieved more than 3 times speedup using "on-demand" transparent huge pages in ClickHouse[1] for a very narrow use-case: random access to a hash table that does not fit in the L3 cache but is not much larger.
But there was a surprise... more than 10 times degradation of overall Linux server performance due to increased physical memory fragmentation after a few days in production: https://github.com/ClickHouse/ClickHouse/commit/60054d177c8bc190e8187c7c5f5e8495a1957a04 https://github.com/ClickHouse/ClickHouse/commit/60054d177c8b...
It was seven years ago, and I hope that the Linux kernel has been improved. I will need to try "revert of revert" of this commit. These changes cannot be tested by microbenchmarks, and only production usage can show their actual impact.
Also, we successfully use huge pages for text section of the executable, and it is beneficial for the stability of performance benchmarks due to lowering the number of iTLB misses.
[1] ClickHouse - high-performance OLAP DBMS: https://github.com/ClickHouse/ClickHouse/ https://github.com/ClickHouse/ClickHouse/
- jeffbee 4y agoThis is why the main feature of a "hugepage aware allocator" is when and how to deallocate, to avoid fragmentation or exploding a collapsed page. https://google.github.io/tcmalloc/temeraire.html https://google.github.io/tcmalloc/temeraire.html