3 ms·
I would recommend reading all the fantastic storage papers coming out of WISC under Remzi and Andrea Arpaci-Dusseau: http://pages.cs.wisc.edu/~remzi/research/
by jorangreef 6y ago
I would recommend reading all the fantastic storage papers coming out of WISC under Remzi and Andrea Arpaci-Dusseau:
http://pages.cs.wisc.edu/~remzi/research/ http://pages.cs.wisc.edu/~remzi/research/
And since you're looking for a GitHub project to watch, I'll go ahead and share TigerBeetle since I'm working on that :) and since we're implementing many of these new storage ideas (disentangling journal corruption from system crashes, protocol-aware recovery for consensus-based storage, hash-chaining to detect misdirected writes and disk corruption, and Direct I/O to recover from the kernel page cache going out of sync with the disk):
https://github.com/coilhq/tigerbeetle https://github.com/coilhq/tigerbeetle
And then finally since you also asked for a YouTube video :) here's a talk we did on Thursday for the Mojaloop Foundation, discussing many of these ideas around storage safety and also around optimizing I/O performance (io_uring, the cost of syscalls and context switches, cache line aligned structures, Direct I/O) and putting this all together into a replicated state machine with strict serializability:
https://www.youtube.com/watch?v=6D7B6v06mBo https://www.youtube.com/watch?v=6D7B6v06mBo