3 ms·
Did you ever down-truncate it? Did your backup program take forever to crunch through non-existent 4TB (and take as much space to restore?) Atomic access wise,
by beagle3 7y ago
Did you ever down-truncate it? Did your backup program take forever to crunch through non-existent 4TB (and take as much space to restore?)
Atomic access wise, that’s fine as long as you don’t map it through a network file system if any sort; also, $DIETY help you recover a consistent state if the power goes out or you kernel panic
Mmap is awesome, I use it all the time. But writing into mmaps robustly is hard. Best example to learn from I am familiar with is symas LMDB source code.
- dekhn 7y agoUnwritten pages in files that are extended don't actually take up space or appear as blocks; a good backup program should respect that. See https://en.wikipedia.org/wiki/Sparse_file https://en.wikipedia.org/wiki/Sparse_file
- beagle3 7y agoIndeed, but many programs, backup or otherwise, or unaware of it. IIRC gzip/bzip/zstd don’t (will spend time reading and writing all the missing pages) whereS gnu tar does.
- terrelln 7y agoZstd by default will write sparse files when it detects runs of zeros during decompression, but isn't aware of it when reading a file.
- butterthebuddha 7y agoWe down-truncated on process exit. I'll take a look at the symas LMDB source code, thanks for pointing that out.