3 ms·
This is basically used for this little gem: http://symas.com/mdb/ http://symas.com/mdb/ LMDB is at the heart of the ubiquitous LDAP ( OpenLDAP ) and is very
by rdtsc 13y ago
This is basically used for this little gem:
http://symas.com/mdb/ http://symas.com/mdb/
LMDB is at the heart of the ubiquitous LDAP ( OpenLDAP ) and is very well optimized ( look at his benchmarks ). Now they are optimized for reading, which is important.
I would imagine mmap-ing with large amount of write will result in unpredictable performance....
- jbert 13y agoYou can try and manage the predictability using msync(), which you'll need anyway if you're doing any kind of transactional write. For more general use, a simple way to keep the amount of 'pending writes' down to reasonable levels is to have a thread walking your mmap()'d regions and enforcing an msync() on a regular basis. This also gives you guarantees such as "each part of mmap'd file is no more than N seconds out of date". Of course, you'll potentially increase disk load overall, since a page dirtied three times, once every second, might only get written out once by the kernel (after 5s) but if you're syncing every 1s it'll get written three times. But you should be able to take control.
- edwintorok 13y agoWrites to mmap-ed regions aren't very reliable: besides the synchronization issue there's also the question of what happens on an I/O error or ENOSPC. Usually your program just gets a SIGBUS and crashes. IIRC LMDB uses regular syscalls to update the mapping, and use mmap only for read.
- hyc_symas 13y agoCorrect, at least for the original design of LMDB. We later added an option to write thru the mmap as well, at the request of some other users. Since LMDB is quite careful about writing and use of fsync/msync, reliability generally isn't an issue, but yes, hard errors on the disk and such can crash your program when using a writable mmap. Also, in the default mode of read-only mmap, the DB is incorruptible, stray pointer writes will crash the program instead of corrupting the DB. With the writable mmap you obviously have no such protection. We recommend developing your app with the default read-only mode, and only enabling the writable mmap after your app code is well tested.