3 ms·
An OS page size is such a prevalent notion in software it's shocking I was oblivious to this a year ago before I got interested in database internals Somethin
by gavinray 4y ago
An OS page size is such a prevalent notion in software it's shocking
I was oblivious to this a year ago before I got interested in database internals
Something that I found interesting, there's a recent presentation by Neumann about the Umbra DBMS where he fields a question about hugepages at the end. I recall him saying they don't use it, which I found interesting.
I know Oracle and MySQL recommended Transparent Hugepages IIRC
- menaerus 4y agoTransparent huge-pages are actually rarely recommended, and I think it's a quite prevalent idiom in database systems world, and that is because they can cause unexpected stalls and latency spikes during the application runtime. This is possible because THB is an in-kernel implementation of system-wide huge-page support and it's basically hard to get a lot of control over the process. OTOH to make use of "normal" huge-pages, you have to allocate them up front so it's not possible to run into THB type of issues. That said, I doubt that enabling huge-pages for complex database workloads, that cannot run solely in-memory, will show any noticeable performance improvement. There's a lot of IO and memory R/W involved and I think this is what shadows the TLB miss cost. What would be interesting, and what I haven't done so far, is to estimate the number of CPU cycles needed for a TLB miss.
- pca006132 4y agoYou need to understand the bottleneck to determine whether or not huge pages are useful. THP requires the kernel to do additional work to give you the huge pages, which is usually more expensive then allocating huge pages through mmap directly.