3 ms·

>> Couldn't you perform a file append without the need for a block erase? > You cannot do single byte writes on NAND. This isn't true - that the OP wrote here

>> Couldn't you perform a file append without the need for a block erase? > You cannot do single byte writes on NAND. This isn't true - that the OP wrote here certainly does work (at least for many places I've tested and used it). I've used it to do all sorts of low level logging in small devices (and I've thoroughly tested many, many chips to complete destruction to see how things degrade). I have not yet seen a place it fails and I've done it on dozens or projects. I guess there may be places wear leveling fights with this, but if you dig into the particular device you have you can usually find low level details to make it work. The places I've pushed it are all commercial projects for which we characterize long term failure by running the flash parts fast enough to destroy the flash, to ensure that once in the field the expected lifetimes meet any guarantees we need to provide. Also, if you've never ran a flash chip to death, try it - it's interesting. Get a small SD card and hammer it withRead/write cycles until you start getting errors, then log and watch how those errors propagate. Good stuff :) Every chip I've tested does indeed let you erase a block (which sets all bits to 0 or 1 depending on chip), then append bits to that block by changing the bits you need. To append, you don't need to update some other data structure, just append a single bit of the correct type, then a bit scan of the page tells you where the data ends. Or, if you have another page telling how many "slots" of yourData are used, again append s single bit. Then small appends only cost a bit. On top of this implement wear leveling (unless the chip does - most bigger ones do, but some low level devices don't). Other projects I've worked on professionally are products for desoldering flash from captured and destroyed enemy devices which are then read raw and reconstructed, fault tolerant and error shielding for NASA space ops with flash, reverse engineering proprietary flash protocols for Secret Service (weird, right? but they pay) uses. I somewhat often get called for consulting on precisely these types of issues. So I do know a bit about low level flash abuse :)

st_goliath 2y ago
I was talking about bare NAND flash, not eMMC or anything otherwise "fancy" with builtin FTL. What you say is certainly true for those. None of the projects you describe sound "cheap" or like they involve devices first-and-foremost optimized for cost, and are unlikely to involve such dirt-cheap, raw NAND devices that are becoming increasingly rare even in the super-cheap embedded space. The original context of the discussion (see: a few comments up) is on-die NAND flash, inside SD cards, which TFA is about. The flash chips I brought up for comparison were from an extremely cost-optimized, embedded device that I worked on ~2016. The more expensive one already being on the upper end, regarding smarts & cost, but still exposing the flash directly (plus ECC & bad block management, but no FTL). I mostly work with industrial devices, and also maintain the Linux user space for raw flash (mtd-utils). I have seen my fair share of broken flash devices over the years (often in-field) and also have a my own little pile of broken industrial SD cards on my desk :)
- SideQuark 2y ago
  > I was talking about bare NAND flash Same here. > None of the projects you describe sound "cheap" or like they involve devices first-and-foremost optimized for cost A lot of them are low cost PIC microcontrollers, which are super cheap, and the lowest cost MCUs were chosen precisely because at large commercial sale scale saving pennies on HW is very important. The reason we went to such lengths to reuse bits at the lowest level is because of needing the lowest cost devices on those projects. Some were higher end, but I think it's more useful on low end chips.
  - st_goliath 2y ago
    > low cost PIC microcontrollers ahhh, I think there's the source of our confusion: I suspect you are actually talking about NOR flash? NOR flash is physically organized different from NAND. It typically does allow single word/byte or sometimes even bit level accesses, like you described. NOR flash is also much more reliable, often not needing any wear leveling at all. The downside is that NOR flash has orders of magnitudes lower data density. Microcontrollers would use NOR flash internally for program memory, precisely because it allows direct, single-cycle access to individual opcodes. With NAND flash, program memory would have to be read page-wise and require some sort of instruction cache in between. Other EEPROM-like devices (e.g. AT28Fxxx) also use NOR flash for pretty much the same reasons. On some embedded Linux devices, NOR flash is used for config/boot loader/kernel, while the rootfs lives on NAND.
    - SideQuark 2y ago
      > ahhh, I think there's the source of our confusion: I suspect you are actually talking about NOR flash? Nope. PIC has on device NOR (which I've tested), and often has external NAND flash (depending on design choices), such as using SDCard interfaces. Here's [1] a Micron tech doc on their NAND chips stating quite clearly they allow exactly what I described doing above: "A NAND Flash block is 128KB. Erasing a block sets all bits to 1 (and all bytes to FFh). Programming is necessary to change erased bits from 1 to 0. The smallest entity that can be programmed is a byte." There's plenty of manufacturers and chip types that guarantee one can do this. It is tech notes like that led me to do some testing to see if devices fall into spec on R/W cycles before death when evaluating which chips to ship in consumer products that need this. [1] https://user.eng.umd.edu/~blj/CS-590.26/micron-tn2919.pdf https://user.eng.umd.edu/~blj/CS-590.26/micron-tn2919.pdf

st_goliath 2y ago

I was talking about bare NAND flash, not eMMC or anything otherwise "fancy" with builtin FTL. What you say is certainly true for those. None of the projects you describe sound "cheap" or like they involve devices first-and-foremost optimized for cost, and are unlikely to involve such dirt-cheap, raw NAND devices that are becoming increasingly rare even in the super-cheap embedded space. The original context of the discussion (see: a few comments up) is on-die NAND flash, inside SD cards, which TFA is about. The flash chips I brought up for comparison were from an extremely cost-optimized, embedded device that I worked on ~2016. The more expensive one already being on the upper end, regarding smarts & cost, but still exposing the flash directly (plus ECC & bad block management, but no FTL). I mostly work with industrial devices, and also maintain the Linux user space for raw flash (mtd-utils). I have seen my fair share of broken flash devices over the years (often in-field) and also have a my own little pile of broken industrial SD cards on my desk :)

SideQuark 2y ago
> I was talking about bare NAND flash Same here. > None of the projects you describe sound "cheap" or like they involve devices first-and-foremost optimized for cost A lot of them are low cost PIC microcontrollers, which are super cheap, and the lowest cost MCUs were chosen precisely because at large commercial sale scale saving pennies on HW is very important. The reason we went to such lengths to reuse bits at the lowest level is because of needing the lowest cost devices on those projects. Some were higher end, but I think it's more useful on low end chips.
- st_goliath 2y ago
  > low cost PIC microcontrollers ahhh, I think there's the source of our confusion: I suspect you are actually talking about NOR flash? NOR flash is physically organized different from NAND. It typically does allow single word/byte or sometimes even bit level accesses, like you described. NOR flash is also much more reliable, often not needing any wear leveling at all. The downside is that NOR flash has orders of magnitudes lower data density. Microcontrollers would use NOR flash internally for program memory, precisely because it allows direct, single-cycle access to individual opcodes. With NAND flash, program memory would have to be read page-wise and require some sort of instruction cache in between. Other EEPROM-like devices (e.g. AT28Fxxx) also use NOR flash for pretty much the same reasons. On some embedded Linux devices, NOR flash is used for config/boot loader/kernel, while the rootfs lives on NAND.
  - SideQuark 2y ago
    > ahhh, I think there's the source of our confusion: I suspect you are actually talking about NOR flash? Nope. PIC has on device NOR (which I've tested), and often has external NAND flash (depending on design choices), such as using SDCard interfaces. Here's [1] a Micron tech doc on their NAND chips stating quite clearly they allow exactly what I described doing above: "A NAND Flash block is 128KB. Erasing a block sets all bits to 1 (and all bytes to FFh). Programming is necessary to change erased bits from 1 to 0. The smallest entity that can be programmed is a byte." There's plenty of manufacturers and chip types that guarantee one can do this. It is tech notes like that led me to do some testing to see if devices fall into spec on R/W cycles before death when evaluating which chips to ship in consumer products that need this. [1] https://user.eng.umd.edu/~blj/CS-590.26/micron-tn2919.pdf https://user.eng.umd.edu/~blj/CS-590.26/micron-tn2919.pdf