8 ms·

"Layering the file system translation on top of the flash translation is inefficient and impedes performance." "For many years SSDs were almost exclusively bui

"Layering the file system translation on top of the flash translation is inefficient and impedes performance." "For many years SSDs were almost exclusively built to seamlessly replace hard drives; they not only supported the same block-device interface" The point of storage is to be able to put anything you want on it. That contract is the block interface, and includes the ability to change the filesystem. A file with internal structures is also a filesystem. The interfaces are fine. Change for change's sake should be avoided. (Providing a bypass, SSD-optimized interface is fine, but, ahem: "put down the crack pipes"... https://news.ycombinator.com/item?id=5541063 https://news.ycombinator.com/item?id=5541063 )

petsos 14y ago
> That contract is the block interface, and includes the ability to change the filesystem. The article argues that we should change this contract. > The interfaces are fine. The article argues that they are not. > Change for change's sake should be avoided. The article argues that we should change them for performance's sake.
- vy8vWJlco 14y ago
  >> Change for change's sake should be avoided. > The article argues that we should change them for performance's sake. I also said providing a bypass was reasonable, but the article gives the impression that the block-level interface is yesterday's jam, and something less than the starting point. It is the starting point and will continue to be because block-level storage is the major use-case. Block-level access accounts for basically all bulk storage in /dev. Extra performance and features (ioctl calls, or a management interface) are gravy, but without the block-level interface, it is not accessible to 99.9% of all software and will not serve for general storage, including pre-existing filesystems, from FAT12 to Btrfs. There are applications for which those are interfaces. There's no reason to make it difficult to apply those layers. Looking forward, if it can't store current and unforseen filesystems out of the box (the block interface), even if that means a 50% reduction in speed, it doesn't deserve the name "mass storage." Noone wants a key-value store even if it is 100% faster. They may be faster, but it doesn't look like storage. And there's no need for it to look different either, because people expect to be able to use it like a block device. Take WD Green drives with their larger allocation sizes. There happens to be an optimal cluster size, but the interface is still that of block IO. If flash works best for a certain allocation size or other tweaks, or even a specific high-level formatting, fine, but it's still going to need to provide the block IO interface as a starting point if it's going to be used as a drive. It's nice to see at least one punitive downvote though.
  - kalleboo 14y ago
    > Block-level access accounts for basically all bulk storage in /dev Unless your home and binary directories are stored on an NFS share.
    - vy8vWJlco 14y ago
      Sure, but even then, on the server - where the storage medium exists, and the bytes are held - it's stored on the operator's choice of filesystem, made possible by the ubiquitous block IO interface (or the platform-specific equivalent thereof). Edit: Network block device, iSCSI, AoE, etc... the block interface is the lifeblood of the storage area network.
baruch 14y ago
I actually agree that using the block interface makes sense for storage but would have loved to have a minimal interference from the SSD. If it just exposed the entire flash to address by the user and reported when a block has problems and maybe some stats about the underlying flash chips I'd be very happy. It would then enable building better things at the OS/Application side. There is quite a bit of a chicken-and-egg problem here though, all the current filesystems basically assume that the underlying media never has any faults and if it does than the media problems are static and do not develop over time. This is obviously incorrect for flash, but even for rotating media it wasn't true. Since every OS requires a fault-free media the SSD vendors are working hard to make sure they provide a semblance of such fault-free media they make it harder to provide the best possible performance or a different trade-off than what they have taken.
- vy8vWJlco 14y ago
  I also want lower-level access, but I would presumably not be using it for files/reliable storage. If a lossy interface is acceptable, why couldn't SSDs simply expose a faster albeit lossy block device and, if necessary, an extended SMART or custom inspection method, and let the user take responsibility for wear-leveling, ECC, etc? It would be backwards compatible with other things by virtue of presenting the standard block interface. A common ECC+wear-leveling middle layer could evolve allowing use of standard filesystems and a common codebase for all flash storage, relieving the apparent burden on SSD vendors who would love to sell fast unreliable storage rather than reliable storage. I think the chicken-egg problem is just an egg problem though, because even though I'd love lower level access to the unreliable bits, I have to expect that the market for unreliable storage is very small, not unlike the high-efficiency-but-sometimes-exploding toilet. :) If reliable storage is the primary use-case, maybe the drives just need to be smarter to keep up. I'd rather an ASIC handle ECC, etc, transparently (for the same reasons I'd rather have a dedicated GPU) than run ECC (or 3D floating point software) on my general processor. If you inevitably want reliable storage and just wind up running ECC, etc, on the CPU, the speed gains disappear and we're back in something similar to a pre-DMA world with the main processor doing something that could be done in parallel by a dedicated chip. If the hard drive is the right place for the offload, I'd rather the economic pressures remain for the SSD vendors to optimize inside that black box, behind the standard reliable interface. That said, again, I too would love finer-grain control.

petsos 14y ago

> That contract is the block interface, and includes the ability to change the filesystem. The article argues that we should change this contract. > The interfaces are fine. The article argues that they are not. > Change for change's sake should be avoided. The article argues that we should change them for performance's sake.

vy8vWJlco 14y ago
>> Change for change's sake should be avoided. > The article argues that we should change them for performance's sake. I also said providing a bypass was reasonable, but the article gives the impression that the block-level interface is yesterday's jam, and something less than the starting point. It is the starting point and will continue to be because block-level storage is the major use-case. Block-level access accounts for basically all bulk storage in /dev. Extra performance and features (ioctl calls, or a management interface) are gravy, but without the block-level interface, it is not accessible to 99.9% of all software and will not serve for general storage, including pre-existing filesystems, from FAT12 to Btrfs. There are applications for which those are interfaces. There's no reason to make it difficult to apply those layers. Looking forward, if it can't store current and unforseen filesystems out of the box (the block interface), even if that means a 50% reduction in speed, it doesn't deserve the name "mass storage." Noone wants a key-value store even if it is 100% faster. They may be faster, but it doesn't look like storage. And there's no need for it to look different either, because people expect to be able to use it like a block device. Take WD Green drives with their larger allocation sizes. There happens to be an optimal cluster size, but the interface is still that of block IO. If flash works best for a certain allocation size or other tweaks, or even a specific high-level formatting, fine, but it's still going to need to provide the block IO interface as a starting point if it's going to be used as a drive. It's nice to see at least one punitive downvote though.
- kalleboo 14y ago
  > Block-level access accounts for basically all bulk storage in /dev Unless your home and binary directories are stored on an NFS share.
  - vy8vWJlco 14y ago
    Sure, but even then, on the server - where the storage medium exists, and the bytes are held - it's stored on the operator's choice of filesystem, made possible by the ubiquitous block IO interface (or the platform-specific equivalent thereof). Edit: Network block device, iSCSI, AoE, etc... the block interface is the lifeblood of the storage area network.

baruch 14y ago

I actually agree that using the block interface makes sense for storage but would have loved to have a minimal interference from the SSD. If it just exposed the entire flash to address by the user and reported when a block has problems and maybe some stats about the underlying flash chips I'd be very happy. It would then enable building better things at the OS/Application side. There is quite a bit of a chicken-and-egg problem here though, all the current filesystems basically assume that the underlying media never has any faults and if it does than the media problems are static and do not develop over time. This is obviously incorrect for flash, but even for rotating media it wasn't true. Since every OS requires a fault-free media the SSD vendors are working hard to make sure they provide a semblance of such fault-free media they make it harder to provide the best possible performance or a different trade-off than what they have taken.

vy8vWJlco 14y ago
I also want lower-level access, but I would presumably not be using it for files/reliable storage. If a lossy interface is acceptable, why couldn't SSDs simply expose a faster albeit lossy block device and, if necessary, an extended SMART or custom inspection method, and let the user take responsibility for wear-leveling, ECC, etc? It would be backwards compatible with other things by virtue of presenting the standard block interface. A common ECC+wear-leveling middle layer could evolve allowing use of standard filesystems and a common codebase for all flash storage, relieving the apparent burden on SSD vendors who would love to sell fast unreliable storage rather than reliable storage. I think the chicken-egg problem is just an egg problem though, because even though I'd love lower level access to the unreliable bits, I have to expect that the market for unreliable storage is very small, not unlike the high-efficiency-but-sometimes-exploding toilet. :) If reliable storage is the primary use-case, maybe the drives just need to be smarter to keep up. I'd rather an ASIC handle ECC, etc, transparently (for the same reasons I'd rather have a dedicated GPU) than run ECC (or 3D floating point software) on my general processor. If you inevitably want reliable storage and just wind up running ECC, etc, on the CPU, the speed gains disappear and we're back in something similar to a pre-DMA world with the main processor doing something that could be done in parallel by a dedicated chip. If the hard drive is the right place for the offload, I'd rather the economic pressures remain for the SSD vendors to optimize inside that black box, behind the standard reliable interface. That said, again, I too would love finer-grain control.