4 ms·

What do you recommend instead?

aappleby 3y ago
We ended up going with a very trivial system based on tar files and bitfield allocators and delegated the more filesystem-y like functionality to a higher level of the stack.
- yjftsjthsd-h 3y ago
  I happen to be in the middle of writing a simple tar implementation, and had in fact just started playing with the idea of just using it as an on-disk filesystem format. The caveats I can see are that 1. it's approximately append-only[0], and 2. you have to replay the whole thing in order to make sure you have the latest version of a file (I'm leaning towards doing so on mount because then I can also build an in-memory filename-to-block-address cache). Have you hit any issues, or is this as surprisingly reasonable as it sounds? [0] I mean, if the new version of a file fits in the same number of 512-byte blocks as the old version you could update in place, but that's an unreliable condition at best and also guarantees data corruption if you lose power halfway; really going append-only makes the implementation simple and also makes it easy to be really resilient.
  - aappleby 3y ago
    I'm not on the team anymore so I don't know if they kept the tar stuff around, but it was straightforward to scan the flash for valid tar header blocks to build the allocation bitmap and then flag header blocks as invalid when files were deleted. The total number of files we needed to keep in the flash was small, so simple brute-force stuff worked fine.
  - e63f67dd-065b 3y ago
    Not so sure about the tar part, but the append-only nature is actually really well-known: look up log-structured filesystems, where all operations append to the log and GC comes to clean up stale data later. I’m in the middle of writing a log-structured disk, and it’s really nice to work with. You get many things like clones, snapshots, etc for free. The replay problem is usually solved with periodic checkpoints of your map, and a checkpoint on shutdown. This way you only ever replay on failure, at which point performance doesn’t matter as much.
    - yjftsjthsd-h 3y ago
      Yeah, knowing there were filesystems designed like that on purpose made me feel better about it:) And once you get over the initial weirdness, the ability to get any version of any file is pretty nice. Though my favorite angle is that if you can live with just burning space[0], it makes the thing so easy to implement:) [0] To be fair, a huge caveat.
  - mschuster91 3y ago
    > The caveats I can see are that 1. it's approximately append-only[0], and 2. you have to replay the whole thing in order to make sure you have the latest version of a file Hmm. I'd go and build a separate metadata/index file that you keep updated on each write(=tar-append) operation. Grow the tar from the beginning of the underlying block device, and the metadata from the rear, and have a pointer in the filesystem header that points to the current metadata address. That way, everything is atomic, you avoid the RAM penalty from having to maintain the metadata mapping, and the only operation that can result in damage in a power-loss event is when the write of the filesystem header messes up - but that can easily be reconstructed by walking both the tar stream and the metadata stream from the beginning to the last element that has a valid header.
  - reycharles 3y ago
    I helped build a tar-backed filesystem for MirageOS[0]. It is definitely easiest to make it append-only. I did add in-place renaming, updates (of file content), pre-allocating a file and removing the last file. We also scan the filesystem on boot and keep a mapping in-memory of file names to file metadata and block offsets, and update this memory representation when modifying the on-disk tar filesystem. If you have a lot of small files this is maybe not a great idea as you will be storing most in memory, though. For the purpose we had in mind it worked fine: A content addressable mirror of package archives. The archives exist somewhere on the web and the package file has a link to the archive and a checksum. We can then download the package and store it by the checksum. This gives integrity that the tar filesystem does not offer. Removing the last file works great if you only have one download job. Otherwise if a download fails and it is not the latest file you can rename it to `failed/<hash>-<random>` and do garbage collection in the filesystem at a later point. [0]: https://github.com/mirage/ocaml-tar/blob/main/mirage/tar_mirage.ml https://github.com/mirage/ocaml-tar/blob/main/mirage/tar_mir... Update: interestingly, this was motivated after trying to use a (mostly) LittleFS-compatible filesystem which unfortunately didn't work very well for us at the time (bugs, poor performance).
crest 3y ago
I would recommend thinking long and hard if you really need a file system on a microcontroller. What do you intend to use the file system for? If it's only for read-only assets like images or sounds use a linker script to collect them into a sorted set. If you want to write logs a ring buffer is faster, simpler and more resilient to corruption. Configuration settings are small enough and don't too frequently in most systems so a ring buffer or just a pair of double buffered flash pages with a checksum and version counter can be enough. LittleFS is tries to hit a sweet spot that only exists on MCUs with NOR flash and relatively much RAM and fast CPU cores. The problem is that it's complex compared to the alternatives. The two most common alternatives I can think of are FAT which will wear out flash given half a chance and SPIFFS which is simpler and slower than LittleFS. FAT is a terrible file system for writing on a MCU, but because of its history as the file system for floppy disks it's still supported by every desktop operating system. This makes it perfect to allow users to edit configuration files stored on a microSD or replace assets like background images without having to use specialised tools. The annoying answer: it depends.
- HocusLocus 3y ago
  Great points. I just want to toss out an example of why you might want an OS to use with flash of decent size to do wear leveling on everything. The classic example I give is a media player subject to power interruption and you don't want to include an internal battery in the design for graceful powerdowns, so the player needs to save a checkpoint of play state every ~5 seconds so the device can resume intuitively. Such as indexing back to your position in a 40 hour audiobook. You're also going to have menu navigation and a combination of changing settings and device local media library changes. And now comes along some Internet angle so there are C libraries that implement browser functions and that means cache and cookie writing. And software updates. Ring buffers are smart but there's a whole lot of writing going on here! It would be great to code everything in high levels so if it emulates properly on a desktop, once it's pushed into the devices (say a generous 128GB flash with 28GB reserved for the system) the devices will have a 20 year lifespan with no batteries to fail or leak.

We ended up going with a very trivial system based on tar files and bitfield allocators and delegated the more filesystem-y like functionality to a higher level of the stack.

yjftsjthsd-h 3y ago
I happen to be in the middle of writing a simple tar implementation, and had in fact just started playing with the idea of just using it as an on-disk filesystem format. The caveats I can see are that 1. it's approximately append-only[0], and 2. you have to replay the whole thing in order to make sure you have the latest version of a file (I'm leaning towards doing so on mount because then I can also build an in-memory filename-to-block-address cache). Have you hit any issues, or is this as surprisingly reasonable as it sounds? [0] I mean, if the new version of a file fits in the same number of 512-byte blocks as the old version you could update in place, but that's an unreliable condition at best and also guarantees data corruption if you lose power halfway; really going append-only makes the implementation simple and also makes it easy to be really resilient.
- aappleby 3y ago
  I'm not on the team anymore so I don't know if they kept the tar stuff around, but it was straightforward to scan the flash for valid tar header blocks to build the allocation bitmap and then flag header blocks as invalid when files were deleted. The total number of files we needed to keep in the flash was small, so simple brute-force stuff worked fine.
- e63f67dd-065b 3y ago
  Not so sure about the tar part, but the append-only nature is actually really well-known: look up log-structured filesystems, where all operations append to the log and GC comes to clean up stale data later. I’m in the middle of writing a log-structured disk, and it’s really nice to work with. You get many things like clones, snapshots, etc for free. The replay problem is usually solved with periodic checkpoints of your map, and a checkpoint on shutdown. This way you only ever replay on failure, at which point performance doesn’t matter as much.
  - yjftsjthsd-h 3y ago
    Yeah, knowing there were filesystems designed like that on purpose made me feel better about it:) And once you get over the initial weirdness, the ability to get any version of any file is pretty nice. Though my favorite angle is that if you can live with just burning space[0], it makes the thing so easy to implement:) [0] To be fair, a huge caveat.
- mschuster91 3y ago
  > The caveats I can see are that 1. it's approximately append-only[0], and 2. you have to replay the whole thing in order to make sure you have the latest version of a file Hmm. I'd go and build a separate metadata/index file that you keep updated on each write(=tar-append) operation. Grow the tar from the beginning of the underlying block device, and the metadata from the rear, and have a pointer in the filesystem header that points to the current metadata address. That way, everything is atomic, you avoid the RAM penalty from having to maintain the metadata mapping, and the only operation that can result in damage in a power-loss event is when the write of the filesystem header messes up - but that can easily be reconstructed by walking both the tar stream and the metadata stream from the beginning to the last element that has a valid header.
- reycharles 3y ago
  I helped build a tar-backed filesystem for MirageOS[0]. It is definitely easiest to make it append-only. I did add in-place renaming, updates (of file content), pre-allocating a file and removing the last file. We also scan the filesystem on boot and keep a mapping in-memory of file names to file metadata and block offsets, and update this memory representation when modifying the on-disk tar filesystem. If you have a lot of small files this is maybe not a great idea as you will be storing most in memory, though. For the purpose we had in mind it worked fine: A content addressable mirror of package archives. The archives exist somewhere on the web and the package file has a link to the archive and a checksum. We can then download the package and store it by the checksum. This gives integrity that the tar filesystem does not offer. Removing the last file works great if you only have one download job. Otherwise if a download fails and it is not the latest file you can rename it to `failed/<hash>-<random>` and do garbage collection in the filesystem at a later point. [0]: https://github.com/mirage/ocaml-tar/blob/main/mirage/tar_mirage.ml https://github.com/mirage/ocaml-tar/blob/main/mirage/tar_mir... Update: interestingly, this was motivated after trying to use a (mostly) LittleFS-compatible filesystem which unfortunately didn't work very well for us at the time (bugs, poor performance).

crest 3y ago

I would recommend thinking long and hard if you really need a file system on a microcontroller. What do you intend to use the file system for? If it's only for read-only assets like images or sounds use a linker script to collect them into a sorted set. If you want to write logs a ring buffer is faster, simpler and more resilient to corruption. Configuration settings are small enough and don't too frequently in most systems so a ring buffer or just a pair of double buffered flash pages with a checksum and version counter can be enough. LittleFS is tries to hit a sweet spot that only exists on MCUs with NOR flash and relatively much RAM and fast CPU cores. The problem is that it's complex compared to the alternatives. The two most common alternatives I can think of are FAT which will wear out flash given half a chance and SPIFFS which is simpler and slower than LittleFS. FAT is a terrible file system for writing on a MCU, but because of its history as the file system for floppy disks it's still supported by every desktop operating system. This makes it perfect to allow users to edit configuration files stored on a microSD or replace assets like background images without having to use specialised tools. The annoying answer: it depends.

HocusLocus 3y ago
Great points. I just want to toss out an example of why you might want an OS to use with flash of decent size to do wear leveling on everything. The classic example I give is a media player subject to power interruption and you don't want to include an internal battery in the design for graceful powerdowns, so the player needs to save a checkpoint of play state every ~5 seconds so the device can resume intuitively. Such as indexing back to your position in a 40 hour audiobook. You're also going to have menu navigation and a combination of changing settings and device local media library changes. And now comes along some Internet angle so there are C libraries that implement browser functions and that means cache and cookie writing. And software updates. Ring buffers are smart but there's a whole lot of writing going on here! It would be great to code everything in high levels so if it emulates properly on a desktop, once it's pushed into the devices (say a generous 128GB flash with 28GB reserved for the system) the devices will have a 20 year lifespan with no batteries to fail or leak.