3 ms·
No, that's a typical size for bcachefs as well. But I've had to optimize for memory usage, performance when working set does not fit into ram - we also don't h
by koverstreet 3y ago
No, that's a typical size for bcachefs as well.
But I've had to optimize for memory usage, performance when working set does not fit into ram - we also don't have a serialize/deserialize step, that will really hurt you when working set doesn't fit in ram.
You want to validate when you're reading in a btree node, but you want to keep that as cheap as possible.
For avoiding the serialize/deserialize, the thing to use now would be Cap'n Proto.
- cperciva 3y agoAh, that would explain it. Yeah, I have a total datastore size which is out of core, but the working set is generally in core... partly because I can use all the RAM on the system if I want to, whereas filesystems are (usually!) expected to occupy just a portion of the system memory. FWIW, my "deserialize" doesn't involve copying all the data -- I just construct pointers into the (serialized) page. Serializing has to copy data though.
- valenterry 3y agoJust wanted to say, I really enjoyed reading your guys conversation, since I barely never do low-level optimizations like that. Very interesting, thanks for sharing your thoughts!