5 ms·
What happens if the program unexpectedly terminates during an update to these persistent data structures? Is the mmap'ed file corrupted, or still in a usable s
by bertr4nd 9y ago
What happens if the program unexpectedly terminates during an update to these persistent data structures? Is the mmap'ed file corrupted, or still in a usable state?
- pjscott 9y agoThe operating system will dutifully write back to disk whatever bytes your program left on those memory-mapped pages at the moment it terminated. If your data structures were in an inconsistent state -- pointers to nowhere, say -- then your file will get in the same inconsistent state, i.e. corrupted.
- dryman 9y agoI don't have a good answer to this question yet. For now I only create the heap in swap, write it to disk, and then use it as read only mmap. To ensure the written file is valid, one can write it to a temporal file first, once confirmed the file is written, then mv the file to desired file name and location. This works for immutable data, but is a big blocker for me to make OPIC work on mutable back store. This problem is generally hard. See [Ensuring data reaches disk](https://lwn.net/Articles/457667/ https://lwn.net/Articles/457667/)
- hyc_symas 9y agoYou should have looked more closely at LMDB, which already solves this problem. Also you could look at LMDB's API for using fixed-address mmap, which lets you store pointer-based structures without any deserialization step at all.