3 ms·
If this filesystem "has comparable random-write performance to an LSM tree", would it be viable to use this filesystem directly as the storage for a key-value s
by derefr 5y ago
If this filesystem "has comparable random-write performance to an LSM tree", would it be viable to use this filesystem directly as the storage for a key-value store (i.e. to swap out LevelDB/RocksDB for a simple library that just creates each key as its own file, expecting to be backed by this filesystem)?
If not, why not? I'm guessing mainly because of kernel context-switching overhead?
And if that's why, then could use of this filesystem be made competitive with [or better-performing than!] e.g. "LevelDB writing to ext4", if that context-switch overhead was removed — e.g. if it was either used by a kernel-mode application (i.e. a unikernel approach); or if the driver itself were moved into userspace as a library, with the expectation that you'd compile it into a single daemon process which would own and have write access to a raw block device?
(I ask because part of my job involves tending to blockchain archive-nodes, and the operational management of LevelDB at scale sometimes makes me want to pull out my hair. A million little 2MB files all in one directory, constantly being created and deleted. If I could 1. work with the keys in those databases directly as a mounted [perhaps read-only] filesystem, and 2. get for free the BetrFS equivalent of Btrfs's incremental subvolume send/receive for them, rather than trying to organize parallel rsync(1) for a million tiny files, those factors alone would be worth dealing with an experimental FS.)
- ww520 5y agoI think the BeTree operation model is simpler than the BeTree based file system. It might be better to use a BeTree database directly.
- derefr 5y agoYeah, that's true. Analogous to using LMDB/BoltDB, directly instead of using Btrfs. And I guess, as long as the BeTree library could understand the concept of a non-expandable database file, you could just point it directly at a raw block device as its "database file" and it would be happy. The only concern I'd have in this case is that userspace database libraries usually don't worry about the possibility of interrupted partial-block writes, since they're usually writing to files on a filesystem, and the filesystem usually handles that possibility for them; while the filesystem itself — or a library working with a raw direct-write block device — does have to worry about partial-block writes. I think, in the case of raw "everything is a tree" storage (B-tree or Be-tree), you'd only need to ensure that 1. there's a journal for root-node-page offsets, and that 2. there's a separate freelist for root-node pages, not hanging off of the root node, but rather attached to the journal; such that root-node pages are only freed for overwrite once the journal entry for the new page is guaranteed flushed to disk. Of course, if you were lazy, you could get the semantics of a journal and rootnode-freelist, by making this database library write its root-node pages as regular sequence-numbered files in a directory backed by a real filesystem, relying on the real filesystem to declare those files fsync(2)'ed before it's willing to delete previous root-node-page files; and then considering the effective freelist to consist only of the intersection of the freelists from all currently-visible root-node-page files.