4 ms·

The problem is that most users (home & enterprise) just want things to work, they don't really care much how to get there and to have the best efficiency. It w

by baruch 14y ago

The problem is that most users (home & enterprise) just want things to work, they don't really care much how to get there and to have the best efficiency. It won't be too hard to have a good filesystem that works over raw NAND flash but it will not work on older OSes, it will not work in the enterprise storage market and so there will be less buyers and thus it will cost more so no one will buy it and it will not be made. Even the enterprise storage folks just want the damn flash devices to just work without the storage folks doing anything with them. It's taken to extremes sometimes and the flash vendors just do whatever they are told since there is a lot of market in whatever the software-defined engineers want. Except the engineers mostly want to deal with high level algorithms and to brag how fast their algorithm is without really thinking about the hardware. Hardware is hard. Besides they can do something with the hardware that is already on the market rather than envision something better. TL;DR unless someone will hold the stick at both ends (software and hardware) no one will make a reduced layer solution.

baruch 14y ago
Case in point for the cost: I looked at buying an OpenSSD (referenced in the article) to write my own firmware to a lowly 64GB device. Was quoted $2000. There is nothing else getting close to this openness on the market.
- adamleventhal 14y ago
  ... and the prognosis for OpenSSD is terminal.
  - baruch 14y ago
    It seems to be geared only towards research insititutes and a specific one at that, most of the work using OpenSSD is concentrated on that one university.
dman 14y ago
Hardware makers make this hard for themselves. Having expensive devkits and requiring NDA's for doing the most trivial things creates a dynamic where the only players able to jump through the monetary and bureaucratic hoops are large vendors. Those vendors are unable to deliver true innovation because they have so much vested in the status quo.
krschultz 14y ago
It may happen in the server world. Engineering wins the day a whole lot more often than in the consumer market. ZFS et al are a good case in point.
lgeek 14y ago
> The problem is that most users (home & enterprise) just want things to work, they don't really care much how to get there and to have the best efficiency. When it comes to research, no one cares that much about what home users and enterprise-users-small-enough-not-to-use-custom-software-stacks want now. Case in point: I don't think many IT mangers were that eager to switch to using ZFS in production when it was announced back in 2004 (and ZFS wad been under development for years at that time). I've considered doing a PhD project pushing and stretching the boundaries of SSD firmware/operating system/filesystems because I think there's a lot of improvement that can be done in this area. The cost of OpenSSD that the sibling comments mention wasn't even that much of a problem. I seriously don't think someone not associated with a research department somewhere would have the time and/or know-how to do original research and implement a working prototype. Hell, I might get a devkit, but I doubt I'll do anything interesting and original at the same time. Which brings us to the actual problem: Documentation and NDAs. For lots of ICs, microcontrollers, processors you can freely get hardware documentation, programming manuals, etc. For flash controllers and high-density NANDs? Almost nothing at all. Maybe some stuff can be reverse engineered and you get some documentation for OpenSSD. But the NAND manufacturer won't tell you stuff that's really, really important about things like failure patterns, which would allow you to optimize error correction and wear leveling for example.
- baruch 14y ago
  Documentation is an issue indeed. I don't think that you really need the inner information about NAND to do the original and innovative research. It really depends on the area you want to work on, the SSD firmware level might require that but the SSD makers are already on that route (some better than others). The other level is to not pay too much attention to the differences between NAND chips and just implement something at a higher level to push the hardware-agnostic smarts to the OS. The OpenSSD also lacks documentation, last time I looked at it there was no info on how to do NCQ on the SATA interface, without which there is no talking about a speedy SSD.

by baruch 14y ago

baruch 14y ago

Case in point for the cost: I looked at buying an OpenSSD (referenced in the article) to write my own firmware to a lowly 64GB device. Was quoted $2000. There is nothing else getting close to this openness on the market.

adamleventhal 14y ago
... and the prognosis for OpenSSD is terminal.
- baruch 14y ago
  It seems to be geared only towards research insititutes and a specific one at that, most of the work using OpenSSD is concentrated on that one university.

dman 14y ago

Hardware makers make this hard for themselves. Having expensive devkits and requiring NDA's for doing the most trivial things creates a dynamic where the only players able to jump through the monetary and bureaucratic hoops are large vendors. Those vendors are unable to deliver true innovation because they have so much vested in the status quo.

krschultz 14y ago

It may happen in the server world. Engineering wins the day a whole lot more often than in the consumer market. ZFS et al are a good case in point.

lgeek 14y ago

> The problem is that most users (home & enterprise) just want things to work, they don't really care much how to get there and to have the best efficiency. When it comes to research, no one cares that much about what home users and enterprise-users-small-enough-not-to-use-custom-software-stacks want now. Case in point: I don't think many IT mangers were that eager to switch to using ZFS in production when it was announced back in 2004 (and ZFS wad been under development for years at that time). I've considered doing a PhD project pushing and stretching the boundaries of SSD firmware/operating system/filesystems because I think there's a lot of improvement that can be done in this area. The cost of OpenSSD that the sibling comments mention wasn't even that much of a problem. I seriously don't think someone not associated with a research department somewhere would have the time and/or know-how to do original research and implement a working prototype. Hell, I might get a devkit, but I doubt I'll do anything interesting and original at the same time. Which brings us to the actual problem: Documentation and NDAs. For lots of ICs, microcontrollers, processors you can freely get hardware documentation, programming manuals, etc. For flash controllers and high-density NANDs? Almost nothing at all. Maybe some stuff can be reverse engineered and you get some documentation for OpenSSD. But the NAND manufacturer won't tell you stuff that's really, really important about things like failure patterns, which would allow you to optimize error correction and wear leveling for example.

baruch 14y ago
Documentation is an issue indeed. I don't think that you really need the inner information about NAND to do the original and innovative research. It really depends on the area you want to work on, the SSD firmware level might require that but the SSD makers are already on that route (some better than others). The other level is to not pay too much attention to the differences between NAND chips and just implement something at a higher level to push the hardware-agnostic smarts to the OS. The OpenSSD also lacks documentation, last time I looked at it there was no info on how to do NCQ on the SATA interface, without which there is no talking about a speedy SSD.