5 ms·
Is there really a need for random 4K writes on the SD card in the Pi? Couldn’t the card use a log-structured filesystem and batch writes in RAM, turning those w
by derefr 7y ago
Is there really a need for random 4K writes on the SD card in the Pi? Couldn’t the card use a log-structured filesystem and batch writes in RAM, turning those writes into streaming writes? The new Pi certainly has enough RAM that it could dedicate some to serving as a write cache.
- pjc50 7y agoThis is the choice that lots of system designers have to make, "low apparent write latency" vs "maximum data at risk of power loss". If there's a database on the Pi you're going to be upset if you call fsync() and it doesn't actually do the writes.
- rowanG077 7y agoif you have implemented fsync() to not sync to actual backing storage you have a bug in your design. That is orthogonal to caching the writes in RAM prior.
- pjc50 7y agoSure, but for this use case if you're doing a set of database transactions each will be its own write to persistent storage. Which is why you care about the random 4k write speed. "Sequential writes are much faster than random writes" is not at all a new problem, it dates from the hard disk era. Which is why most filesystems will cache as much as they think they can get away with.
- cmurf 7y agoWhen I do OS updates (Fedora, so that's dnf and rpm) with a system on USB flash or SD Card flash, the write are abysmal. Like, 3-4MB/s, using `iotop -d3 -o`. There's almost no reads suggesting the RPMs that have been downloaded are cached. I experience this with ext4, btrfs, and f2fs. Using strace on both dnf and rpm, there's no appreciable amount of sync(), fsync(), or fdatasync() - so I'm not really sure why there was this persistent ~4MB/s trickle to the card rather than large "state" updates, in particular for f2fs and Btrfs. All three make heavy use of delayed allocation, but in particular f2fs and Btrfs have no specifically designated locations for write, or overwrites. So I'd expect that most update OS type writes should/could be much faster. A sequential write is ~20MB/s on both of these media, and sequential reads are ~150MB/s. I wasn't ever able to profile it well enough to understand why it's so dog slow.
- hedora 7y agoThis is a false dichotomy. Most (all?) contemporary high-reliability (and, arguably, high performance) filesystems are long structured. (I am excluding things such as ext4 that are missing data block checksumming, and recovery from disk failure using parity stripes)
- gruez 7y ago>high-reliability (and, arguably, high performance) filesystems are long structured. That prevents the filesystem from being in a inconsistent state, but doesn't address the issue of data loss. eg. what happens if there's 5M outstanding writes and you get a fysnc? If you wait for everything to be flushed, that's going to produce latency. If you lie and return immediately, and there's a power outage, you now lost those oustanding writes.
- hedora 7y agoA decent FS will make sure there are never 5M outstanding writes that an unrelated fsync depends on.
- geerlingguy 7y agoIf there's ever a way to avoid writing small chunks to disk, that's preferred. But I think people are spoiled by having fast SSDs in their local machines, and much of today's software, web applications, etc. expect to be able to treat SSDs as RAM, pretty much. Once I can fully boot my Pis off USB, I'm planning on reworking my cluster to run off SSD alone to see what kind of performance I can get out of it.