4 ms·

No, that is actually not what the Flash Adaptation Layer does. The FAL's primary job is to make the Flash array look like a disk, despite the fact that individ

by phkamp 4y ago

No, that is actually not what the Flash Adaptation Layer does. The FAL's primary job is to make the Flash array look like a disk, despite the fact that individual "sectors" are not individually rewriteable. To do this, the FAL implements which is for all practical purposes a filesystem, where the files are all a single sector long and where the filename is the sector number exposed by the "disk". In other words: Now we have two filesystems on top of each other, one lying to the other, which does a lot of unnecessary work, because it is being lied to.

kazinator 4y ago
s/for all practical purposes/for the sake of rhetorical convenience in my argument/ > , the FAL implements which is for all practical purposes a filesystem, where the files are all a single sector long and where the filename is the sector number exposed by the "disk". 1. That is not a "file system" comparable to the thing above which you're also calling "file system", which means you're essentially equivocating on the term. 2. Any old magnetic hard drive exposes this same file system: it makes equal-sized sectors available under names that are indices. There is no lookup structure that is not a filesystem under your definition.
- phkamp 4y ago
  On a magnetic disk, the exposed "name" is the physical address, on a SSH device, the data is stored ${somewhere} and the FAL has a (pretty interesting!) datastructure to lookup were that is, when you demand a certain "name". When you write a certain "name", the magnetic disk just overwrites what is already there, at the assigned spot. When you write a certain "name" on a SSD, a new spot gets allocated (This may require reshuffling akin to garbage collect), and the data structure is updated for the new locations ("=$name") and old location ("unused"), and if making the old location unused means that an entire eraseblock becomes free, then erasing that is scheduled, after which it is added to the free pool. But that is only the easy part. The hard part is "wear levelling", which essentially amounts to erasing all the blocks approximately the same amount of times, because that is what wears out the gate material in the individual cells. Wear levelling involves taking data which the host has not asked for, copying to a different location, in order to empty out the erase-block with the least wear (= erase cycles) so that it can shoulder more of the load. And now comes the truly hideous part: The FAL has to do this in a way where it's own metadata is part of it's own wear-levelling, this is why most competent FAL's have a basic structure much like a classic Log-structured filesystem. So yes: We do have two filesystems on top of each other, and exposing a more suited model than "array of equal-sized rewritable sectors" could reduce that.
  - deleted 4y ago
    [deleted]
  - StillBored 4y ago
    Ignoring the fact that just about all the HD's made since the 1980's when IDE became popular can relocate sectors, and since the 2000's didn't even necessarily expose the physical sector sizes and all kinds of other abstractions. So, the idea that we shouldn't have a disk abstraction to allow actual filesystems to focus on what matters to the user is sorta nonsense. You probably have this idea that all flash disks are the same, and i'm here to tell you they are not, just like not all computers are 8 cores big.little machines. Disks scale from little tiny emmc, controllers to large shared arrays that are doing deduplication, replication, etc,etc,etc and the media can be individual flash chips, massive parallel flash, spinning disks, arrays of spinning disks, mixes of flash and spinning disk, etc, etc, etc. There have been a half dozen raw flash layers in linux over the past ~15 years, and generally they all suck, make assumptions about the flash that doesn't hold for more than a couple generations, end up slower than off the shelf flash with FTL's (aka what your calling FAL), and have frequently failed to allow the kinds of filesystem layering that one expects of linux/grub/etc. (and then there are the ones running at various hyperscalers that consume more RAM than most people have in their PC's).
    - phkamp 4y ago
      Bad block remapping is just a trivial table lookup, the first disk drives did it with 8-bit microcontrollers. In my experience a good FAL is about as hard to write as a good filesystem. While you can parameterize a lot of things, there are some fundamental properties of the flash cells which make it very hard to write a single FAL which works well with all flash chips. As a matter of principle I do not comment on issues specific to Linux.
      - deleted 4y ago
        [deleted]
      - kazinator 4y ago
        > is just a trivial table lookup So just an inch to the left goalpost of the True Scotsman's "filesystem" definition. > make it very hard to write a single FAL which works well with all flash chips Right? So the last thing you want is to foist that logic into filesystems. The layered separation is good. Hence, someone upthread wrote: A flash adaptation layer solves the following problem: I have M filesystems, that I'd like to use on any one of N different flash technologies. I don't want to complicate each M filesystem with support for N flashes.

by phkamp 4y ago

kazinator 4y ago

s/for all practical purposes/for the sake of rhetorical convenience in my argument/ > , the FAL implements which is for all practical purposes a filesystem, where the files are all a single sector long and where the filename is the sector number exposed by the "disk". 1. That is not a "file system" comparable to the thing above which you're also calling "file system", which means you're essentially equivocating on the term. 2. Any old magnetic hard drive exposes this same file system: it makes equal-sized sectors available under names that are indices. There is no lookup structure that is not a filesystem under your definition.

phkamp 4y ago
On a magnetic disk, the exposed "name" is the physical address, on a SSH device, the data is stored ${somewhere} and the FAL has a (pretty interesting!) datastructure to lookup were that is, when you demand a certain "name". When you write a certain "name", the magnetic disk just overwrites what is already there, at the assigned spot. When you write a certain "name" on a SSD, a new spot gets allocated (This may require reshuffling akin to garbage collect), and the data structure is updated for the new locations ("=$name") and old location ("unused"), and if making the old location unused means that an entire eraseblock becomes free, then erasing that is scheduled, after which it is added to the free pool. But that is only the easy part. The hard part is "wear levelling", which essentially amounts to erasing all the blocks approximately the same amount of times, because that is what wears out the gate material in the individual cells. Wear levelling involves taking data which the host has not asked for, copying to a different location, in order to empty out the erase-block with the least wear (= erase cycles) so that it can shoulder more of the load. And now comes the truly hideous part: The FAL has to do this in a way where it's own metadata is part of it's own wear-levelling, this is why most competent FAL's have a basic structure much like a classic Log-structured filesystem. So yes: We do have two filesystems on top of each other, and exposing a more suited model than "array of equal-sized rewritable sectors" could reduce that.
- deleted 4y ago
  [deleted]
- StillBored 4y ago
  Ignoring the fact that just about all the HD's made since the 1980's when IDE became popular can relocate sectors, and since the 2000's didn't even necessarily expose the physical sector sizes and all kinds of other abstractions. So, the idea that we shouldn't have a disk abstraction to allow actual filesystems to focus on what matters to the user is sorta nonsense. You probably have this idea that all flash disks are the same, and i'm here to tell you they are not, just like not all computers are 8 cores big.little machines. Disks scale from little tiny emmc, controllers to large shared arrays that are doing deduplication, replication, etc,etc,etc and the media can be individual flash chips, massive parallel flash, spinning disks, arrays of spinning disks, mixes of flash and spinning disk, etc, etc, etc. There have been a half dozen raw flash layers in linux over the past ~15 years, and generally they all suck, make assumptions about the flash that doesn't hold for more than a couple generations, end up slower than off the shelf flash with FTL's (aka what your calling FAL), and have frequently failed to allow the kinds of filesystem layering that one expects of linux/grub/etc. (and then there are the ones running at various hyperscalers that consume more RAM than most people have in their PC's).
  - phkamp 4y ago
    Bad block remapping is just a trivial table lookup, the first disk drives did it with 8-bit microcontrollers. In my experience a good FAL is about as hard to write as a good filesystem. While you can parameterize a lot of things, there are some fundamental properties of the flash cells which make it very hard to write a single FAL which works well with all flash chips. As a matter of principle I do not comment on issues specific to Linux.
    - deleted 4y ago
      [deleted]
    - kazinator 4y ago
      > is just a trivial table lookup So just an inch to the left goalpost of the True Scotsman's "filesystem" definition. > make it very hard to write a single FAL which works well with all flash chips Right? So the last thing you want is to foist that logic into filesystems. The layered separation is good. Hence, someone upthread wrote: A flash adaptation layer solves the following problem: I have M filesystems, that I'd like to use on any one of N different flash technologies. I don't want to complicate each M filesystem with support for N flashes.