6 ms·

I am looking forward to the day when all SSDs ship minimal firmware, and offload all the complex work to (main-CPU) software.

rwmj 11y ago
Unfortunately that day isn't going to happen. In fact the opposite is happening -- interesting spinning rust technology like Shingled Magnetic Recording (SMR) drives are getting more complex translation layers.
- deleted 11y ago
  [deleted]
- Turing_Machine 11y ago
  I wouldn't say that it's never going to happen based on current trends. This kind of thing tends to be quasi-cyclic. At one time, desktop computers had separate FPUs. Over time, those got integrated onto the main CPU, but then the trend switched back, with heavy calculations moving off-CPU to the GPU. The mainframe->desktop->cloud story is similar. I say "quasi"-cyclic because (e.g.) cloud isn't really the same as mainframe, but the transition back and forth between centralized and decentralized is still striking nonetheless.
  - marssaxman 11y ago
    I'm eagerly awaiting the part of the cycle where the GPU gets folded back into the processor. I'm really not interested in writing code against mystery-meat architectures with no documented ABI...
    - gcr 11y ago
      We're already seeing that with integrated, "soldered-on" graphics cards, and with frameworks like OpenCL, and higher-level abstractions like Theano, Torch7, etc.
      - marssaxman 11y ago
        All we need now is documentation for the low-level abstractions so we can scrap all the vendor code and compile our own GPU machine language.
    - greggyb 11y ago
      I am not sure based on reading your post whether or not I am providing new information to you: Keep an eye on the AMD APUs moving forward and their HSA progress.
      - marssaxman 11y ago
        Thanks, that sounds like an interesting development.
nine_k 11y ago
SSDs that use SATA aren't going to allow any finer control, I suspect, due to the interface's limitations. PCIe SSD devices, though, already feature an advanced controller (that is, a CPU core). This core could be programmed from the "computer" side somehow — but building any public API is always a major commitment, and takes much more effort than a proprietary API which you can change at will.
Retric 11y ago
In the long term latency makes that a non starter.
wtallis 11y ago
Having the flash translation layer on the drive itself is the only practical way for the drive to be usable by multiple operating systems. Almost everyone who uses a SSD wants it to be accessible to at least two operating systems (UEFI, and whatever OS resides on the SSD). Doing the FTL on the host system would basically relegate the SSD to being just a cache device.
- zokier 11y ago
  We have these things called standards which allow nice things like interoperability. Besides, you do not need the entire drive to be accessible from UEFI, it is enough that you can load a small bootloader which contains the drivers to handle the disk. Thirdly, UEFI has extensible right in the name, shouldn't be too hard for OSes to throw a disk driver blob into UEFI.
  - bduerst 11y ago
    Doesn't that require partitioning the disk? That doesn't seem like an OOB solution.
    - wtallis 11y ago
      It's not the requirement to partition the drive that kills the idea. Setting aside the first small chunk of the drive for the firmware to read is how it's always been done. The problem is that this would require the partitioning to be done below the wear leveling layer, which reduces the effectiveness of the wear leveling slightly and means you can only change your bootloader settings a thousand times before the drive is dead.
      - pgeorgi 11y ago
        sadly with NAND even reads contribute to wear.
      - Dylan16807 11y ago
        That's not really what kills it. You can use a small ring buffer to boost that to 100 thousand, and/or use the flash in a more reliable way there (like SLC or devoting 2/3 of the bits to ECC).
VLM 11y ago
The user experience of win-modems, win-printers, and win-scanners was absolutely horrific so you really need to click edit and add the sarcasm tag.
ebbv 11y ago
This is a bad idea for a lot of reasons. A well designed system has components that just work and have standardized interfaces. What you're suggesting is that each device should be totally unusable unless the machine has the specific software running that knows how to use it. That's poor design.
- zanny 11y ago
  Thats how... pretty much every peripheral in existance works. You need webcam drivers. Video card drivers. Sound card drivers. USB drivers, at the least for generic HID devices. Nothing stops there from being common standard generic SSD drivers, where vendors could publish their own if they had an optimization. Right now the SATA interface prevents anything like that from happening, and the market has already moved to these proprietary broken controllers with braindead GL-class protocols to access them. (SATA, NVMe)
  - wtallis 11y ago
    Any such interface for SSDs would need to be able to work with a dozen different kinds of flash, a few sizes for each, a wide range of quantities of flash chips, all while exposing the performance and endurance characteristics of the flash and the topology of its connection to the host bus so that the software FTL can properly tune itself. You're asking for a hardware interface of unprecedented complexity and cross-platform vendor neutral drivers of comparable complexity to 3d graphics drivers, all for the sake of letting SSDs have "dumb" controllers for a generation or two before the technology changes enough that the controllers need to start translating things again.
AYBABTME 11y ago
Wouldn't that make performance a lot worse, for both CPU and disks? Offloading the job to something closer to the flash chips and fully dedicated to servicing them sounds like a better (and the current) path. Same for high speed network links.
- rwmj 11y ago
  This is appropriate: http://www.catb.org/jargon/html/W/wheel-of-reincarnation.html http://www.catb.org/jargon/html/W/wheel-of-reincarnation.htm...
  - AYBABTME 11y ago
    Always have that in the back of my mind =]. However I don't think that's the current trend right now. Except where special hardware is integrated to the chip, but the work isn't left to the CPU itself. I'm not so versed in the area so I'm just blind guessing here.
- wtallis 11y ago
  Flash translation layers don't really need to be offloaded for the sake of performance. You need some kind of controller to multiplex access to a dozen or more flash chips from a narrow PCIe link, so you might as well throw in an ARM core to do the translation and wear leveling and garbage collection, and then you can stick a few supercaps in the drive too and have a robust hardware platform.
- wmf 11y ago
  The FTL runs much faster on a 3GHz Xeon core than on a 150MHz ARM core. Of course, the cost and power consumption is also much higher.
- bbulkow 11y ago
  FusionIO cards run with FTL maintenance on the main CPU. Run a workload and you'll see the driver spinning in a way that isn't accounted for by your operations. It's not pretty - at high workloads, the network interrupt traffic starts interfering with all the bus traffic to the cards. A controller can be designed to run exactly at the speed of the chips, no more no less, leaving the CPU doing what it does best. That being said, if you build your system correctly, there is not any extra CPU load created by maintaining the FTL at app level. The main problem is the lack of an API for wear detection. I have discussed this with senior chip designers at the major manufacturers, and they are all loath to nail down to one api, as they believe how wear is specified will have to change over time. That's a broad brush - some manufacturers are more open to these discussions than others. I'm still working on them.
nqzero 11y ago
this would be a huge win for database design - FTL is incredibly wasteful in terms of unnecessary block erases for cache
sliverstorm 11y ago
Welcome back to driver hell. We thought you weren't ever coming back, but here you are!
moonbug 11y ago
If you think that, I've a crate of ST-506s to sell you.

rwmj 11y ago

Unfortunately that day isn't going to happen. In fact the opposite is happening -- interesting spinning rust technology like Shingled Magnetic Recording (SMR) drives are getting more complex translation layers.

deleted 11y ago
[deleted]
Turing_Machine 11y ago
I wouldn't say that it's never going to happen based on current trends. This kind of thing tends to be quasi-cyclic. At one time, desktop computers had separate FPUs. Over time, those got integrated onto the main CPU, but then the trend switched back, with heavy calculations moving off-CPU to the GPU. The mainframe->desktop->cloud story is similar. I say "quasi"-cyclic because (e.g.) cloud isn't really the same as mainframe, but the transition back and forth between centralized and decentralized is still striking nonetheless.
- marssaxman 11y ago
  I'm eagerly awaiting the part of the cycle where the GPU gets folded back into the processor. I'm really not interested in writing code against mystery-meat architectures with no documented ABI...
  - gcr 11y ago
    We're already seeing that with integrated, "soldered-on" graphics cards, and with frameworks like OpenCL, and higher-level abstractions like Theano, Torch7, etc.
    - marssaxman 11y ago
      All we need now is documentation for the low-level abstractions so we can scrap all the vendor code and compile our own GPU machine language.
  - greggyb 11y ago
    I am not sure based on reading your post whether or not I am providing new information to you: Keep an eye on the AMD APUs moving forward and their HSA progress.
    - marssaxman 11y ago
      Thanks, that sounds like an interesting development.

nine_k 11y ago

SSDs that use SATA aren't going to allow any finer control, I suspect, due to the interface's limitations. PCIe SSD devices, though, already feature an advanced controller (that is, a CPU core). This core could be programmed from the "computer" side somehow — but building any public API is always a major commitment, and takes much more effort than a proprietary API which you can change at will.

Retric 11y ago

In the long term latency makes that a non starter.

wtallis 11y ago

Having the flash translation layer on the drive itself is the only practical way for the drive to be usable by multiple operating systems. Almost everyone who uses a SSD wants it to be accessible to at least two operating systems (UEFI, and whatever OS resides on the SSD). Doing the FTL on the host system would basically relegate the SSD to being just a cache device.

zokier 11y ago
We have these things called standards which allow nice things like interoperability. Besides, you do not need the entire drive to be accessible from UEFI, it is enough that you can load a small bootloader which contains the drivers to handle the disk. Thirdly, UEFI has extensible right in the name, shouldn't be too hard for OSes to throw a disk driver blob into UEFI.
- bduerst 11y ago
  Doesn't that require partitioning the disk? That doesn't seem like an OOB solution.
  - wtallis 11y ago
    It's not the requirement to partition the drive that kills the idea. Setting aside the first small chunk of the drive for the firmware to read is how it's always been done. The problem is that this would require the partitioning to be done below the wear leveling layer, which reduces the effectiveness of the wear leveling slightly and means you can only change your bootloader settings a thousand times before the drive is dead.
    - pgeorgi 11y ago
      sadly with NAND even reads contribute to wear.
    - Dylan16807 11y ago
      That's not really what kills it. You can use a small ring buffer to boost that to 100 thousand, and/or use the flash in a more reliable way there (like SLC or devoting 2/3 of the bits to ECC).

VLM 11y ago

The user experience of win-modems, win-printers, and win-scanners was absolutely horrific so you really need to click edit and add the sarcasm tag.

ebbv 11y ago

This is a bad idea for a lot of reasons. A well designed system has components that just work and have standardized interfaces. What you're suggesting is that each device should be totally unusable unless the machine has the specific software running that knows how to use it. That's poor design.

zanny 11y ago
Thats how... pretty much every peripheral in existance works. You need webcam drivers. Video card drivers. Sound card drivers. USB drivers, at the least for generic HID devices. Nothing stops there from being common standard generic SSD drivers, where vendors could publish their own if they had an optimization. Right now the SATA interface prevents anything like that from happening, and the market has already moved to these proprietary broken controllers with braindead GL-class protocols to access them. (SATA, NVMe)
- wtallis 11y ago
  Any such interface for SSDs would need to be able to work with a dozen different kinds of flash, a few sizes for each, a wide range of quantities of flash chips, all while exposing the performance and endurance characteristics of the flash and the topology of its connection to the host bus so that the software FTL can properly tune itself. You're asking for a hardware interface of unprecedented complexity and cross-platform vendor neutral drivers of comparable complexity to 3d graphics drivers, all for the sake of letting SSDs have "dumb" controllers for a generation or two before the technology changes enough that the controllers need to start translating things again.

AYBABTME 11y ago

Wouldn't that make performance a lot worse, for both CPU and disks? Offloading the job to something closer to the flash chips and fully dedicated to servicing them sounds like a better (and the current) path. Same for high speed network links.

rwmj 11y ago
This is appropriate: http://www.catb.org/jargon/html/W/wheel-of-reincarnation.html http://www.catb.org/jargon/html/W/wheel-of-reincarnation.htm...
- AYBABTME 11y ago
  Always have that in the back of my mind =]. However I don't think that's the current trend right now. Except where special hardware is integrated to the chip, but the work isn't left to the CPU itself. I'm not so versed in the area so I'm just blind guessing here.
wtallis 11y ago
Flash translation layers don't really need to be offloaded for the sake of performance. You need some kind of controller to multiplex access to a dozen or more flash chips from a narrow PCIe link, so you might as well throw in an ARM core to do the translation and wear leveling and garbage collection, and then you can stick a few supercaps in the drive too and have a robust hardware platform.
wmf 11y ago
The FTL runs much faster on a 3GHz Xeon core than on a 150MHz ARM core. Of course, the cost and power consumption is also much higher.
bbulkow 11y ago
FusionIO cards run with FTL maintenance on the main CPU. Run a workload and you'll see the driver spinning in a way that isn't accounted for by your operations. It's not pretty - at high workloads, the network interrupt traffic starts interfering with all the bus traffic to the cards. A controller can be designed to run exactly at the speed of the chips, no more no less, leaving the CPU doing what it does best. That being said, if you build your system correctly, there is not any extra CPU load created by maintaining the FTL at app level. The main problem is the lack of an API for wear detection. I have discussed this with senior chip designers at the major manufacturers, and they are all loath to nail down to one api, as they believe how wear is specified will have to change over time. That's a broad brush - some manufacturers are more open to these discussions than others. I'm still working on them.

nqzero 11y ago

this would be a huge win for database design - FTL is incredibly wasteful in terms of unnecessary block erases for cache

sliverstorm 11y ago

Welcome back to driver hell. We thought you weren't ever coming back, but here you are!

moonbug 11y ago

If you think that, I've a crate of ST-506s to sell you.