3 ms·

Someone needs to write a book about breaking into writing software like RocksDB, HSE, etc. Years ago I found myself wanting to learn more however going from 0 t

Someone needs to write a book about breaking into writing software like RocksDB, HSE, etc. Years ago I found myself wanting to learn more however going from 0 to 1 felt impossible. Graduated from a T3 school in CS so understanding the concepts wasn't the issue, I just didn't know how to build a good foundation in low latency persistence. Years later I ended up contributing to low latency java which was really interesting, but what an opportunity missed.

nn3 6y ago
"How can I learn an instrument without listening to music" "How can I learn to write stories without reading books?" Have you considered just reading the code? It's all available. Best way to learn is to look what the masters are doing. And even complex software is typically just a collection of simple things together. IMHO one of the biggest failing of many CS courses is that they never get above toy software. Would be much better to at least once dive into some production software and try to figure something out in a real code base.
- krapht 6y ago
  Reading the code of a piece of production software is a poor way to get understanding of the fundamentals of a strange topic. It's really inefficient in terms of "understanding" per hour; the student has no context as to why certain choices were made for what reason, just that they were made.
sahil-kang 6y ago
I just finished reading the OSTEP book[1] and it has a nice chapter on SSDs[2]. The entire last portion of the book is about filesystems/disks so you might find it interesting. [1] http://pages.cs.wisc.edu/~remzi/OSTEP/ http://pages.cs.wisc.edu/~remzi/OSTEP/ [2] http://pages.cs.wisc.edu/~remzi/OSTEP/file-ssd.pdf http://pages.cs.wisc.edu/~remzi/OSTEP/file-ssd.pdf
- organicfigs 6y ago
  Oh my god this book is amazing! Thank you
- wtallis 6y ago
  That chapter on SSDs looks pretty good to me. Their numbers for NAND page and especially erase block sizes are very outdated; more modern values are 4kB to 16kB for NAND pages and 16-24MB for erase blocks on TLC NAND. Section 44.9 on mapping table sizes is a little bit odd, because most SSDs really do have 1GB of RAM per 1TB of flash, and that expense is widely seen as worthwhile even for multi-TB SSDs. The exceptions are low-end consumer SSDs that cache only part of the mapping table in a smaller amount of DRAM or SRAM, and a few enterprise/datacenter models that use 32kB block sizes for their FTL instead of the typical 4kB and thus reduce the DRAM requirement by a factor of 8 at the expense of greatly lowered performance and increased write amplification when writing in units smaller than 32kB. Aside from the two above issues, everything looks correct and relevant, and I can't think of any missing details that deserve to be added to an introduction of that length.
- ignoramous 6y ago
  > pages.cs.wisc.edu/~remzi/OSTEP I see a maintenance page instead: "Sorry! The URL you requested was not found on our server. Is it Sunday between 4 and 8 PM (CST)? If so, the server may be undergoing regularly scheduled downtime. Otherwise, please contact the maintainer of the referring page and ask them to fix their link. Thanks!" Here's a Jan, 2020 snapshot: https://web.archive.org/web/20200122013800/https://pages.cs.wisc.edu/~remzi/OSTEP/ https://web.archive.org/web/20200122013800/https://pages.cs....

nn3 6y ago

"How can I learn an instrument without listening to music" "How can I learn to write stories without reading books?" Have you considered just reading the code? It's all available. Best way to learn is to look what the masters are doing. And even complex software is typically just a collection of simple things together. IMHO one of the biggest failing of many CS courses is that they never get above toy software. Would be much better to at least once dive into some production software and try to figure something out in a real code base.

krapht 6y ago
Reading the code of a piece of production software is a poor way to get understanding of the fundamentals of a strange topic. It's really inefficient in terms of "understanding" per hour; the student has no context as to why certain choices were made for what reason, just that they were made.

sahil-kang 6y ago

I just finished reading the OSTEP book[1] and it has a nice chapter on SSDs[2]. The entire last portion of the book is about filesystems/disks so you might find it interesting. [1] http://pages.cs.wisc.edu/~remzi/OSTEP/ http://pages.cs.wisc.edu/~remzi/OSTEP/ [2] http://pages.cs.wisc.edu/~remzi/OSTEP/file-ssd.pdf http://pages.cs.wisc.edu/~remzi/OSTEP/file-ssd.pdf

organicfigs 6y ago
Oh my god this book is amazing! Thank you
wtallis 6y ago
That chapter on SSDs looks pretty good to me. Their numbers for NAND page and especially erase block sizes are very outdated; more modern values are 4kB to 16kB for NAND pages and 16-24MB for erase blocks on TLC NAND. Section 44.9 on mapping table sizes is a little bit odd, because most SSDs really do have 1GB of RAM per 1TB of flash, and that expense is widely seen as worthwhile even for multi-TB SSDs. The exceptions are low-end consumer SSDs that cache only part of the mapping table in a smaller amount of DRAM or SRAM, and a few enterprise/datacenter models that use 32kB block sizes for their FTL instead of the typical 4kB and thus reduce the DRAM requirement by a factor of 8 at the expense of greatly lowered performance and increased write amplification when writing in units smaller than 32kB. Aside from the two above issues, everything looks correct and relevant, and I can't think of any missing details that deserve to be added to an introduction of that length.
ignoramous 6y ago
> pages.cs.wisc.edu/~remzi/OSTEP I see a maintenance page instead: "Sorry! The URL you requested was not found on our server. Is it Sunday between 4 and 8 PM (CST)? If so, the server may be undergoing regularly scheduled downtime. Otherwise, please contact the maintainer of the referring page and ask them to fix their link. Thanks!" Here's a Jan, 2020 snapshot: https://web.archive.org/web/20200122013800/https://pages.cs.wisc.edu/~remzi/OSTEP/ https://web.archive.org/web/20200122013800/https://pages.cs....