9 ms·
Littlefs – A little fail-safe filesystem designed for microcontrollers
- robinsoh 5y agodon't most embedded devices use emmc now? with the wear leveling under the control of the mmc device rather than raw NAND or NOR?
- matja 5y agoAnd there's no specification on how that wear-leveling works (or doesn't work).
- hwh 5y agoThat is not true. See https://github.com/littlefs-project/littlefs/blob/master/DESIGN.md https://github.com/littlefs-project/littlefs/blob/master/DES... Sorry, as a commenter on this pointed out, the comment I was replying to was probably talking about eMMC. For which indeed no wear levelling is specified by the JEDEC standard - it is a standard that defines the communication protocol between a device and a host system. Newer JEDEC standards define the interface to access vendor specific "health reports", the actual method of wear levelling isn't regularly disclosed by vendors.
- phs2501 5y agoI think the GP was commenting on the lack of specification of eMMC wear leveling, not Littlefs.
- stjohnswarts 5y agoSandisk has multiple documents on this. "They" is a pretty big paintbrush.
- kevin_thibedeau 5y agoYou never know if a marginal brand has licensed the Sandisk patents or implemented them correctly.
- iamjackg 5y agoI don't know about higher level, more expensive stuff, but many popular microcontrollers (like the ESP8266 and ESP32) have SPI flash onboard, without any built-in wear leveling.
- pantalaimon 5y agoYou are thinking Linux class devices, this is about MCUs with a few MiB of external flash
- lloydatkinson 5y agoI haven't had chance to use this yet but it's on my list to use with an AVR. Does anyone have any experience using it, how is it? I really like how it's unit tested and seems fairly well documented and even has github actions setup to build and test which seems to be rare for C libraries on github.
- kevin_thibedeau 5y agoIt is a nice library. It's backend api is simple to hook into whatever storage scheme you choose to use. It's also nice to be able to disable wear leveling and use it as a fault tolerant private filesystem from an image file.
- gsempe 5y agoIt's very nice to get your experience feedback. Thank you very much. I'm writing Embedsys Weekly newsletter and so I have listed littlefs project for the next issue. Do you think you can eventually share to the reader your experience in few words? If you feel good with the idea, my email is embedsys@embedsysweekly.com
- matt_trentini 5y agoIt's one of the filesystem options in MicroPython and we've used it extensively for a number of (medical device) products. It's been bullet-proof for us.
- gsempe 5y agoHi Matt, I just proposed the exact same thing to a sibling commenter. I plan to add Littlefs to the next issue of Embedsys Weekly. If you feel confortable to share your experience with my readers on 3/4 sentences it would be fantastic.
- gjadi 5y agoAt my work, we are using littlefs on several products, it works quite well. The v1.X have performance issues when you have a lot of small files. But the v2.X solve them.
- kstenerud 5y agoI was a little disconcerted at the size of lfs.c, but then again there are advantages to having it all in a single file when compiling for a microcontroller. Although now I wonder why the crc32 code is the only thing in a separate file?
- TonyTrapp 5y agoThe latest commit message on that file is "Changed lfs_crc to match more common API (...)" so I would assume that they factored out this function specifically so that it could be easily exchanged with another implementation. There are many CRC32 implementations that may be more suitable for the specific microcontroller you are targetting, or maybe you already have an CRC32 implementation in your codebase that you would like to reuse because space is scarce. For the other functions the reusability isn't really given.
- magicalhippo 5y agoAs an example, most if not all STM32's have CRC32 hardware as well, so by pulling it out one can use such a hardware implementation.
- NonEUCitizen 5y agoThey could've put an ifdef/#ifndef around it and not have an extra file.
- SteveMoody73 5y agoYou could do that but if you want to support a lot of different microcontrollers which may all have different implementations, separating the code into a different file makes more sense. Even if you add ifdef statements for every supported controller in there and the file becomes a mess of conditional code, you still have a clean implementation of the file system.
- deleted 5y ago[deleted]
- roland35 5y agoThis is pretty cool! It looks like a good option for Flash memory. If anyone is looking for a FAT file system (ie for an SD card or for making a USB mass storage device) fatfs is still a great option.
- matt_trentini 5y agoJust beware that FATFS is very easy to corrupt in the event of power outages - unless it's read-only. Personally I would be reluctant to use FATFS as a writable filesystem on a commercial embedded device.
- pantalaimon 5y agoUnfortunately LittleFS is very slow on large (multi-GB) storage like SD cards. Also SD cards are managed flash, so you don't need to do wear leveling in the FS.
- Inviz 5y agoYeah, littlefs wear leveling can be turned off for this case. Protection against corruption, and posix like api at small size is still useful.
- pantalaimon 5y agoUh I didn’t know that - have you run it on an SD card successfully? What options did you use?
- kevin_thibedeau 5y agoYou set block_cycles to -1 to disable wear leveling and have the erase callback be a NOP that just returns success. Other geometry parameters can be whatever suits you.
- baybal2 5y agoBSD* 3 clause license, which means Freertos kernel 10.0+ only. The prime majority of SDKs from chip vendors still use much older Freertos ports which were under GPL RelianceFS is GPL as well * Typo fix
- kstenerud 5y agoMIT license is compatible with GPL.
- alwyn 5y agoIt looks like BSD-3-Clause to me, not MIT?
- stjohnswarts 5y agoWhat about BSD keeps you from including this in linux? It is compatible with GPL2 & 3. There is some BSD code in the kernel and a ton in Linux utilities.
- unwind 5y agoThe source files seem to state Copyright (c) 2017, Arm Limited. All rights reserved. so this is from Arm? That is not so clearly communicated, but perhaps given the license (standard BSD 3-Clause) I guess it doesn't really matter. Might be politically sensitive in some organizations, but perhaps I'm just being a cynic. Looks really nice, and stuff like this is seriously useful and can really make engineering on embedded platforms way easier.
- brudgers 5y agoThe organization has one person who is also a member of Arm Embed, so it seems to be tied to Arm.
- Inviz 5y agoJust integrated littlefs into my stm32 framework. Author has a helper library for coroutines, which can be used to plug in asynchronous hooks, making the fs non-blocking. My next step is to run SQLite on it.
- hungryforcodes 5y agoSQLite on an STM32
- chme 5y agoSTM32 is a very broad range of chips. STM32MP1 runs Linux and I am sure SQLite as well ;) And even leaving the MP1 aside, the STM32H7 micro controller has a ARM Cortex-M7 core which runs up to 550MHz.
- qbasic_forever 5y agoJust wild to me that a 'little' microcontroller for a few bucks is more powerful than any computer I had up to about 1998 or so.
- kevin_thibedeau 5y agoTake solace that your phone would have been a TOP500 supercomputer in the early 90s.
- winrid 5y agoHurts even more when you were using a 450mhz PIII in 2006...
- exikyut 5y agoHA, same here!! IBM 300PL by any chance? Mine (256MB, 8GB) ran Windows 98, and (record scratch) a copy of Visual Basic 1.0 that I'd stumbled on an abandonware copy of while attempting to figure the internet out at the local library. lol Then there was the the 486 with 4MB RAM and 8MB free disk that I was playing with just a year earlier (yep - 2005). And the 800MHz AMD Duron w/ 320MB RAM I used between 2012-2014. And now there's a 2004-era Pentium 4 in the corner here that I'm currently running ddrescue on to try and shoo away ~300KB of unhappy disk sectors. What can I say... work-blocking health issues guarantees life will be InTeReStInG haha Still playing catchup to equilibrium, and taking longer than I'd like because where it matters I annoyingly go a little slower than everyone else (huff). I'm almost certain you were probably doing something slightly more interesting on your PIII...? Delphi, maybe, or perhaps a newer (interestinger) version of VB? Or maybe a super-lightweight Linux setup and Vim? (Oh now that reminds me of the old Toshiba laptop I found on the side of the road with a ~300MHz K6-2 and (IIRC) 32MB RAM in it... hibernation worked so well on that thing it's how I would typically boot into Slackware on it (4 second resume time on a 20 year old laptop?!?! yes please!). Sadly my install CD was a bit scratched so the only things that really worked were bash, grep and sed... which I attempted to write a package manager for Slackware in. I got halfway there too, but had to quit because the 15-line regexes I was writing were giving me a headache. However, to this day I have zero issue with regular expressions up to sed's level of complexity. I do badly need to go back and revisit the subject though.)
- danuker 5y agoOf 545 commits, 193 mention "fix" or "fixed" or "bug". That is 35.4%, quite high, compared to 10 large C projects I analyzed in 2019, which have a median of 25%. That could mean file systems are hard to create, or that this particular one is buggy. https://danuker.go.ro/frequency-of-bugfix-commits.html https://danuker.go.ro/frequency-of-bugfix-commits.html Edit: looks like it's a problem with other file systems also: https://github.com/pellepl/spiffs https://github.com/pellepl/spiffs - 40.2% https://github.com/tytso/e2fsprogs https://github.com/tytso/e2fsprogs - 36.8% But a reimplementation of exFAT is not as buggy: https://github.com/relan/exfat https://github.com/relan/exfat - 19.4%
- userbinator 5y agoBut a reimplementation of exFAT is not as buggy That's not surprising, because the FAT family is relatively simple for a filesystem. As a result, you can easily get read-only support done almost right the first time, and writing is only slightly more difficult.
- bonyt 5y agoIs it a problem, or is it that filesystems are not something you need to add new features to as often and thus commits are biased towards fixes? Or some other reason for a bias?
- Rexxar 5y agoOr, maybe, it just indicates they prioritize fixing bugs over adding new buggy features.
- authorityofnil 5y agoIn addition, the size of patches can also vary quite a lot on what the author considers a "thing". Some developers fix multiple things at once, while some others think every nut and bolt should be it's own separate commit
- stjohnswarts 5y agoYes I believe in save early save often and don't bother squashing and have multiple commits usually in my branch before a merge request. We have another team member that religiously squashes everything down to a single commit on his feature branch before requesting a pull from his branch. We don't have micromanagers so that's just the way things are.
- m4lvin 5y agoThis is used in InfiniTime, the PineTime smart watch firmware :-) See https://github.com/InfiniTimeOrg/InfiniTime/pull/438 https://github.com/InfiniTimeOrg/InfiniTime/pull/438
- sitzkrieg 5y agoive used this in an embedded system that had the power ripped out of it constantly (sorta like automotive battery powered) and it has held up for years in the field. the only failure so far was an sdcard finally going out
- dang 5y agoRelated: The design of littlefs: A fail-safe filesystem designed for microcontrollers - https://news.ycombinator.com/item?id=20845112 https://news.ycombinator.com/item?id=20845112 - Aug 2019 (72 comments) A little fail-safe filesystem designed for microcontrollers (2017) - https://news.ycombinator.com/item?id=18989705 https://news.ycombinator.com/item?id=18989705 - Jan 2019 (30 comments)
- pjsg 5y agoThe problem with this filesystem is that it doesn't actually protect against power fails. Mostly you are OK, but not always. I run an afl-base fuzzer against it back in 2020, and found a bunch of issues leading to assert failures in the code. I wasn't even checking data integrity. The tickets are still open (and it wasn't obvious to me how to fix them). Spiffs is another micro controller file system, but it also fails fuzzing tests very rapidly with various failures. Spiffs seems less maintained than littlefs. Are there any other good, reliable, freely licensed, microcontroller file systems suitable for small flash parts?
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- pvarangot 5y agoI used it at work twice and it was mostly ok with power failures, thing is you have to be careful if you are mounting it very early on boot if your embedded OS can bootloop fast after LFS is already mounted, because if you nuke it with a flurry of power fails it will break. For example if you have a battery you probably want to be sure you have a reliable measurement for SoC and enough juice to stay up before you mount LFS.
- bsder 5y ago> The problem with this filesystem is that it doesn't actually protect against power fails. Mostly you are OK, but not always. Power failure is a tough problem in embedded file systems (aka--too small for Linux). Effectively--you need to be "append-only with checksums at checkpoints". Every one of those words is anathema to making your flash small and cheap.