4 ms·
LMDB is a really great read optimized key-value, transactional key-value store, but you have to be careful with when benchmarking memory mapped files. The probl
by crest 4y ago
LMDB is a really great read optimized key-value, transactional key-value store, but you have to be careful with when benchmarking memory mapped files. The problem is that a thread touching paged out memory just blocks without performing any system call. It looks like you're faster and it starts out very fast because as long as your data fits into main memory you can read it without any context switches, but as soon as you get under even slight memory pressure jitter goes through the roof and your userspace code is helpless because there is no load-if-mapped instruction to avoid blocking the thread if your accessed page has to be read in from the backing storage. Your database has no control over I/O parallelism and queue depth except very indirectly by delegating database accesses to a thread pool of a specific size and hoping for the best.
- malkia 4y agotrue dat, there was an excellent series from one University going over that in details - changed my opinion on the matter - https://db.cs.cmu.edu/mmap-cidr2022/ https://db.cs.cmu.edu/mmap-cidr2022/