3 ms·

While what you’re saying is valid, what you classify as bullshit (log-structured storage) is still immensely popular and important in the world of flash. In my

While what you’re saying is valid, what you classify as bullshit (log-structured storage) is still immensely popular and important in the world of flash. In my experience, read-ahead is still extremely important, even when you have SSDs, and unless all your writes are >4kb, you’re still going to benefit from a certain amount of “sequential-ness” of your writes. Optimizing things for disk storage still does pay of tremendously, it’s just that the way things are optimized are very different, and perhaps much more nuanced. Where before it was just as easy as saying “just read/write things sequential, random access is expensive”, nowadays you need to think about how the kernel interacts with storage, and sector sizes and whatnot. There definitely are do’s and don’ts when optimizing data structures for this.

xani_ 4y ago
If you're writing on SATA SSD, yes. If you're writing on NVMe, well, there is a good chance that if you're not at the high end (big site, a lot of things to do), you can just ignore that as the IOPS will be "good enough" Like, single relatively shitty NVMe can still sustain ~700MB/s of random(4k block) writes. Use good ones, and use more than one and you quickly hit CPU barrier before you hit NVMe performance. You still want to keep it kinda grouped together but that's mostly for the wear levelling reasons
- formercoder 4y ago
  Doesn’t the chip on the drive deal with wear leveling?
  - xani_ 4y ago
    It does but it's always easier if you allocate/deallocate big blocks. Especially if you just put sub-block-size data on disk
- stingraycharles 4y ago
  A lot of the cloud is based on iSCSI, which, although “random access” time is relatively constant, still has an increased latency compared to local storage. As such, you definitely do want to benefit from things such as read-ahead, and designing your storage structures around these types of mechanisms yields significantly better results.

xani_ 4y ago

If you're writing on SATA SSD, yes. If you're writing on NVMe, well, there is a good chance that if you're not at the high end (big site, a lot of things to do), you can just ignore that as the IOPS will be "good enough" Like, single relatively shitty NVMe can still sustain ~700MB/s of random(4k block) writes. Use good ones, and use more than one and you quickly hit CPU barrier before you hit NVMe performance. You still want to keep it kinda grouped together but that's mostly for the wear levelling reasons

formercoder 4y ago
Doesn’t the chip on the drive deal with wear leveling?
- xani_ 4y ago
  It does but it's always easier if you allocate/deallocate big blocks. Especially if you just put sub-block-size data on disk
stingraycharles 4y ago
A lot of the cloud is based on iSCSI, which, although “random access” time is relatively constant, still has an increased latency compared to local storage. As such, you definitely do want to benefit from things such as read-ahead, and designing your storage structures around these types of mechanisms yields significantly better results.