9 ms·

Why SATA is now obsolete

163 points by mrkd 12y ago

Article based on paper: http://research.cs.wisc.edu/adsl/Publications/yiying-thesis13.pdf

vortico 12y ago
I like "news" articles like this. Harris takes a scientific publication, adds his own opinion while making the topic easier to understand, and cites his source.
- elif 12y ago
  Personally, i believe his armchair dismissal of FTL requires far more justification than he gives. This blog post is pure postulation that doesn't even attempt to take up the argument of the dissertation, let alone take that argument far enough to justify that title. In fact in the thesis' abstract, the proposed solution is still a layer of indirection. Instead of the FTL giving an abstracted address space, the "device" is given the responsibility of controlling wear, gc, and parallelism by dynamically reallocating blocks itself, and the filesystem picking up some more of the management stuff. I am certainly not qualified enough to read this dissertation critically enough to know if it is fundamentally sound theoretically, but as a scientist I know that it is close to meaningless until it has been experimentally tested and reviewed..
mrkd 12y ago
Article based on paper: http://research.cs.wisc.edu/adsl/Publications/yiying-thesis13.pdf http://research.cs.wisc.edu/adsl/Publications/yiying-thesis1...
- gwern 12y ago
  "De-indirection for Flash-based Solid State Drives", Zhang 2013 http://research.cs.wisc.edu/adsl/Publications/yiying-thesis13.pdf http://research.cs.wisc.edu/adsl/Publications/yiying-thesis1... "Flash-based solid-state drives (SSDs) have revolutionized storage with their high performance. Modern flash-based SSDs virtualize their physical resources with indirection to provide the traditional block interface and hide their internal operations and structures. When using a file system on top of a flash-based SSD, the device indirection layer becomes redundant. Moreover, such indirection comes with a cost both in memory space and in performance. Given that flash-based devices are likely to continue to grow in their sizes and in their markets, we are faced with a terrific challenge: How can we remove the excess indirection and its cost in flash-based SSDs? We propose the technique of de-indirection to remove the indirection in flashbased SSDs. With de-indirection, the need for device address mappings is removed and physical addresses are stored directly in file system metadata. By doing so the need for large and costly indirect tables is removed, while the device still has its freedom to control block-allocation decisions, enabling it to execute critical tasks such as garbage collection and wear leveling. In this dissertation, we first discuss our efforts to build an accurate SSD emulator. The emulator works as a Linux pseudo block device and can be used to run real system workloads. The major challenge we found in building the SSD emulator is to accurately model SSDs with parallel planes. We leveraged several techniques to reduce the computational overhead of the emulator. Our evaluation results show that the emulator can accurately model important metrics for common types of SSDs, which is sufficient for the evaluation of various designs in this dissertation and in SSD-related research. Next, we present Nameless Writes, a new device interface that removes the need for indirection in flash-based SSDs. Nameless writes allow the device to choose the location of a write; only then is the client informed of the name (i.e., address) where the block now resides. We demonstrate the effectiveness of nameless writes by porting the Linux ext3 file system to use an emulated nameless-writing device and show that doing so both reduces space and time overheads, thus making for simpler, less costly, and higher-performance SSD-based storage. We then describe our efforts to implement nameless writes on real hardware. Most research on flash-based SSDs including our initial evaluation of nameless writes rely on simulation or emulation. However, nameless writes require fundamental changes in the internal workings of the device, its interface to the host operating system, and the host OS. Without implementation in real devices, it can be difficult to judge the true benefit of the nameless writes design. Using the OpenSSD Jasmine board, we develop a prototype of the Nameless Write SSD. While the flash-translation layer changes were straightforward, we discovered unexpected complexities in implementing extensions to the storage interface. Finally, we discuss a new solution to perform de-indirection, the File System De-Virtualizer (FSDV), which can dynamically remove the cost of indirection in flash-based SSDs. FSDV is a light-weight tool that de-virtualizes data by changing file system pointers to use device physical addresses. Our evaluation results show that FSDV can dynamically reduce indirection mapping table space with only small performance overhead. We also demonstrate that with our design of FSDV, the changes needed in file system, flash devices, and device interface are small."
sp332 12y ago
That sounds fine in principle, but please don't tell me the future is jffs2!
- wmf 12y ago
  I think the proposal is something like ZFS/Btrfs but with even more complexity to handle wear leveling and such.
  - etcet 12y ago
    Is this going to have a net benefit? To my mind it makes more sense for specialized functionality to be built into the drives firmware rather than letting the CPU deal with it.
Someone1234 12y ago
The title is odd. Should be "Why SATA is now obsolete." The article is of course correct, and some manufacturers are already offering SSD-like storage which is connected like a stick of RAM. I'm sure the gap between storage and the main system pipeline (CPU-GPU-RAM-etc) will only shrink as time moves forward. However as an interim solution things that acted like hard drives were convenient.
- hga 12y ago
  But there are also SAS (Serial Attached SCSI) SSDs ... the major point is in the aphorism in his third paragraph: "All problems in computer science can be solved by another level of indirection, except for the problem of too many layers of indirection." Hence this level of indirection, the discrete "drive", is what he thinks should go poof.
  - jobu 12y ago
    > "All problems in computer science can be solved by another level of indirection, except for the problem of too many layers of indirection." First time I've heard that quote, but I love it. Reminds me of this one: "There are only two hard problems in Computer Science: cache invalidation, naming things, and off-by-one errors."
    - Already__Taken 12y ago
      These are way better than that bloody binary joke in every computer department.
      - leeoniya 12y ago
        the better version of the binary joke is Binary: as easy as 1, 10, 11
        Someone1234 12y ago
        As easy as 3, 2, 1? I realise you read binary from right to left, not left to right, but the sentence is in English so you read that left to right. Also the commas and binary mixed together are a little odd. Least of all because you didn't make all three two binary digits (i.e. 11, 10, 01). Regardless that joke is going to cause more arguments than laughs.
blt 12y ago
Linkbait title is linkbait, but content is sound.
jkot 12y ago
> The obvious question is: why maintain two logical address spaces? There are perhaps 10 more address spaces and translations. It the way OS and hardware are build. > Why not have the file system directly manage the flash? Too slow innovation, it takes decade to replace operating system. Also main CPU is very power hungry and slow... More likely the SSD controller will become part of CPU and SOC.
KaiserPro 12y ago
"why maintain two logical address spaces? Why not have the file system directly manage the flash?" Because managing flash is hard. For this to work, the amount of exceptions in the filesystem it's self would make it unmanageable. an example is this: a Fusion-io card up until recently used low quality flash, yet managed to get stella performance. Much higher performance than the equivalent intel ssd card (who have access to much higher quality flash) some of that is down to layout, the rest firmware. Then you have the flash that lives on a device like your phone, however the firmware of the SoC is tweaked to run that particular flash controller. At its heart, flash/hdd is just a key:value database (presented with 4k value size) why would we want to complicate that (sure we need to indicate to the hardware that we've deleted a page, but to be honest not much else)
- gcb0 12y ago
  same was told about cylinders and sectors a couple decades ago. to use your example: imagine if you could use the fusion-io FS on the intel flash. also remember that several SSD companies that even had their own flash fab are now gone because their firmware was crap.
- rasz_pl 12y ago
  >Then you have the flash that lives on a device like your phone, however the firmware of the SoC is tweaked to run nowadays all of the things (ha) use flash behind higher level of abstraction, be it emmc, UFS, or SD controller. There are no phone soc firmware tweaks.
- stcredzero 12y ago
  At its heart, flash/hdd is just a key:value database (presented with 4k value size) why would we want to complicate that To avoid an inherent bottleneck?. Perhaps we'd be better served by a larger number of key:value stores with greater parallelism?
  - wmf 12y ago
    SSDs are already parallel and I don't think updating the single FTL mapping table is a bottleneck.
stcredzero 12y ago
now that non-volatile memory technology - flash today, plus RRAM tomorrow - has been widely accepted, it is time to build systems that use flash directly instead of through our antique storage stacks. Hear hear! The basic storage model that arose in the 50's and 60's being the best fit for recently developed hardware seems as likely as the security models from the 50's and 60's being the best fit for the Internet. (If the obnoxious popup adverts from ZDNet strike me as archaic, they are not doing well.)
- jschwartzi 12y ago
  If NAND continues to function the way it does today, we would still need to standardize on some kind of interface to get traction. The advantage of SATA, SAS, PCIe, et. al. are their pervasiveness in the PC world. His proposal basically boils down to "NAND on SATA is kinda slow because of all the latency from stuff. We should just hook the NAND up to one of the upstream buses." It's unclear to me how this is different from what is already being done.
Animats 12y ago
The author is on to something. Flash memory devices can emulate disks, but that's an inefficient way to access them, especially for reading. You have all the overhead of Linux system calls and file systems to, basically, read a RAM-like device that's orders of magnitude faster in access time than a rotating device. The question is, what non-file abstraction should a flash device offer? Slow persistent RAM? A key/value store? An SQL database? Computing has had "RAM" and "disks" as its storage models for so long that those concepts are nailed into software at a very low level. The NoSQL crowd could probably find uses for devices that implemented big, fast key/value stores.
- wmf 12y ago
  Some of my coworkers bypassed the OS for Redis, although the FTL is still there. http://public.dhe.ibm.com/common/ssi/ecm/po/en/pos03135usen/POS03135USEN.PDF http://public.dhe.ibm.com/common/ssi/ecm/po/en/pos03135usen/...
  - sanderjd 12y ago
    Funny how your comment told me more about what that offering is supposed to do than the whole Solution Brief did! All the same, very neat stuff.
- dfox 12y ago
  The problem with NAND flash is that it is not "RAM-like device" even for reads (smallest addressable entity is block even for reads). It can essentially do three operations: read block, write block, erase bunch of blocks at once. For the purposes of filesystem, this is actually an workable storage interface that does not need any additional abstraction layers.
  - robotresearcher 12y ago
    You are both right. It's a RAM-like key-value store with large (block-sized) values.
    - 12y ago
danbruc 12y ago
One word. Separation of concerns. Okay, three. Why would you want to mix managing files and transistors?
jwatte 12y ago
Yes, the SATA interface to flash storage is obsolete. Just like internal combustion engines! And it will be almost as hard to get rid of.
- wtallis 12y ago
  SATA and AHCI are already on their way out, being replaced by PCIe and NVMe. That removes most of the abstraction that is actually obsolete, but it doesn't pretend that SSDs aren't fundamentally block devices. About the only thing NVMe does that is perpetuating a fiction about the underlying flash is that the read block size and write block size are the same thing. All the rest is abstraction that's absolutely necessary until we completely change our conventions for computer architecture and operating systems.
mark-r 12y ago
I've always wondered how much performance gain you could get if the very large blocks required for erasure were known to the file system, rather than being hidden behind an emulation layer.
ebbv 12y ago
It sounds like a great idea to "get rid of abstraction" and have the file system handle the NAND directly if you don't think about it. But if you think about it for even a little bit you realize that you can't just get rid of the abstraction, you're just moving it out of the drive's firmware and into the file system. And that gets you nothing because now the file system has to be compatible with every manufacturer's NAND drives. And the file system has to be updated any time that system needs to be changed. You can't just pop a new NAND drive in and have it work, you have to download new NAND drivers for the file system, etc. I'm also skeptical of just how much performance gain you'd see. The article doesn't include these figures, which makes me skeptical that they are significant. Delegation of duties is important for good system design. Drives necessarily need to present themselves in a standard way to the system. Having SSDs appear like normal drives is fine. They aren't memory and they shouldn't be used as memory.
- sanxiyn 12y ago
  Flash Translation Layer is critical to performance. Googling "flash translation layer performance" should be enough to convince you of this. Moving it from firmware to file system does not improve performance by itself, but it enables optimizations. And FTLs in firmware are far from optimal.
rwmj 12y ago
If you're interested in this subject from a Linux/storage expert, here's Ric Wheeler's talk from FOSDEM last year: https://archive.fosdem.org/2014/schedule/event/persistent_memory/ https://archive.fosdem.org/2014/schedule/event/persistent_me...
jordanpg 12y ago
I'm not a hardware expert, but my intuition tells me that the general thesis -- that any given hardware module could be implemented in some way that's light years ahead of where we are now -- is often true. But there is more to widespread adoption than being the best technology: marketing, timing, industrial realities, legacy software limitations... Legacy enterprise software and OSs in particular seem they probably make significant architectural changes to the very core of every computer difficult to effect.
lbenes 12y ago
As a related note, if you upgrade to a SSD be sure to change the SATA type in BIOS from IDE to AHCI. Not only did I see a doubling in write speed [1], but after making the change, my system felt much more responsive under heavy load. [1] http://forum.crucial.com/t5/Crucial-SSDs/Why-do-i-need-AHCI-with-a-SSD-Drive-Guide-Here-Crucial-AHCI-vs/td-p/57078 http://forum.crucial.com/t5/Crucial-SSDs/Why-do-i-need-AHCI-...
gumby 12y ago
Basically the author is calling for a return to the Multics file system, where files were just a way of referring to capability-managed subsets of an enormous pool of pages. I liked the idea the first time around so I like it again. But it's harder than it sounds. I find something lovely and poetic in the fact that Lisp is over 50 years old and Multics is almost that old, and both may be having a bit of a renaissance.
rocky1138 12y ago
This isn't an entirely novel idea. I remember in 2009 Steve Wozniak giving a talk in Waterloo about becoming the Chief Scientist for Fusion-io (https://fusionio.com/ https://fusionio.com/), a company building PCI-E SSDs, bypassing the SATA bus.
- pyromine 12y ago
  This is somewhat tangential, but my professor recruited Steve to come to Fusion-io and always speaks incredibly highly of Steve. When he was leaving the CEO position to give it back to the founder of Fusion-io he offered Steve some additional shares and Steve told him to give the shares the engineers who actually produce the technology. Incredible humbleness there.
- pyromine 12y ago
  This is somewhat tangential, but my professor recruited Steve to come to Fusion-io and always speaks incredibly highly of Steve. When he was leaving the CEO position to give it back to the founder of Fusion-io he offered Steve some additional shares and Steve told him to give the shares the engineers who actually produce the technology. Incredible humbleness there.
- pyromine 12y ago
  This is somewhat tangential, but my professor recruited Steve to come to Fusion-io and always speaks incredibly highly of Steve. When he was leaving the CEO position to give it back to the founder of Fusion-io he offered Steve some additional shares and Steve told him to give the shares the engineers who actually produce the technology. Incredible humbleness there.
WaxProlix 12y ago
Anyone else think it's a little ironic to read about something being obsolete on ZDNet?
markbnj 12y ago
I think the author misunderstands the word "obsolete." Something is only obsolete when there is something better to replace it with.
duckingtest 12y ago
My SSD has a three core cpu dedicated to it. Removing the layer of indirection isn't going to make it faster. At best it's going to make the ssd cheaper by a minuscule amount due to a simpler controller.
- wmf 12y ago
  One core of your main processor is faster than those three embedded cores; that's why Fusion-io mooches off the main processor.
  - duckingtest 12y ago
    Its probably fast enough for its purpose. If it isn't, making it faster is way easier than changing existing abstraction layers. In the extreme, it can ever be an asic.
markhahn 12y ago
the basic premise is that remapping is necessarily expensive. unfortunately, neither the thesis nor its citations really support that. and we have plenty of examples where remapping is pervasive and apparently efficient (MMUs, for instance). it's hard to imagine how a filesystem could reasonably handle wear-levelling, not to mention the now-common practice of using MLC cells to hold SLC data for faster writes, etc. the "nameless writes" idea is OK - a FS person would just call it even lazier allocation, not anything really different.
zobzu 12y ago
all he means behind the sensationalistic crap is that the sata controller is not actually needed, its there for compatibility. "hard disk" will not be the ram. its just is made of the same chips, but still logically separated. and there is no need for a sata controller to address it really. That's all. You still need a file system if you want to address the files. Just like address blocks of ram or anything else. If you give free reign to the app, there is no security, reliability, etc. Even ram is actually indexed as well. On top of that you need a representation for the human behind it - no matter how hard they try to remove that (because its much easier to lock you in when there is no generic way to browser a memory device), its not very convenient compared to a file system.
deleted 12y ago
[deleted]

163 points by mrkd 12y ago

Article based on paper: http://research.cs.wisc.edu/adsl/Publications/yiying-thesis13.pdf

I like "news" articles like this. Harris takes a scientific publication, adds his own opinion while making the topic easier to understand, and cites his source.

elif 12y ago
Personally, i believe his armchair dismissal of FTL requires far more justification than he gives. This blog post is pure postulation that doesn't even attempt to take up the argument of the dissertation, let alone take that argument far enough to justify that title. In fact in the thesis' abstract, the proposed solution is still a layer of indirection. Instead of the FTL giving an abstracted address space, the "device" is given the responsibility of controlling wear, gc, and parallelism by dynamically reallocating blocks itself, and the filesystem picking up some more of the management stuff. I am certainly not qualified enough to read this dissertation critically enough to know if it is fundamentally sound theoretically, but as a scientist I know that it is close to meaningless until it has been experimentally tested and reviewed..

mrkd 12y ago

Article based on paper: http://research.cs.wisc.edu/adsl/Publications/yiying-thesis13.pdf http://research.cs.wisc.edu/adsl/Publications/yiying-thesis1...

gwern 12y ago
"De-indirection for Flash-based Solid State Drives", Zhang 2013 http://research.cs.wisc.edu/adsl/Publications/yiying-thesis13.pdf http://research.cs.wisc.edu/adsl/Publications/yiying-thesis1... "Flash-based solid-state drives (SSDs) have revolutionized storage with their high performance. Modern flash-based SSDs virtualize their physical resources with indirection to provide the traditional block interface and hide their internal operations and structures. When using a file system on top of a flash-based SSD, the device indirection layer becomes redundant. Moreover, such indirection comes with a cost both in memory space and in performance. Given that flash-based devices are likely to continue to grow in their sizes and in their markets, we are faced with a terrific challenge: How can we remove the excess indirection and its cost in flash-based SSDs? We propose the technique of de-indirection to remove the indirection in flashbased SSDs. With de-indirection, the need for device address mappings is removed and physical addresses are stored directly in file system metadata. By doing so the need for large and costly indirect tables is removed, while the device still has its freedom to control block-allocation decisions, enabling it to execute critical tasks such as garbage collection and wear leveling. In this dissertation, we first discuss our efforts to build an accurate SSD emulator. The emulator works as a Linux pseudo block device and can be used to run real system workloads. The major challenge we found in building the SSD emulator is to accurately model SSDs with parallel planes. We leveraged several techniques to reduce the computational overhead of the emulator. Our evaluation results show that the emulator can accurately model important metrics for common types of SSDs, which is sufficient for the evaluation of various designs in this dissertation and in SSD-related research. Next, we present Nameless Writes, a new device interface that removes the need for indirection in flash-based SSDs. Nameless writes allow the device to choose the location of a write; only then is the client informed of the name (i.e., address) where the block now resides. We demonstrate the effectiveness of nameless writes by porting the Linux ext3 file system to use an emulated nameless-writing device and show that doing so both reduces space and time overheads, thus making for simpler, less costly, and higher-performance SSD-based storage. We then describe our efforts to implement nameless writes on real hardware. Most research on flash-based SSDs including our initial evaluation of nameless writes rely on simulation or emulation. However, nameless writes require fundamental changes in the internal workings of the device, its interface to the host operating system, and the host OS. Without implementation in real devices, it can be difficult to judge the true benefit of the nameless writes design. Using the OpenSSD Jasmine board, we develop a prototype of the Nameless Write SSD. While the flash-translation layer changes were straightforward, we discovered unexpected complexities in implementing extensions to the storage interface. Finally, we discuss a new solution to perform de-indirection, the File System De-Virtualizer (FSDV), which can dynamically remove the cost of indirection in flash-based SSDs. FSDV is a light-weight tool that de-virtualizes data by changing file system pointers to use device physical addresses. Our evaluation results show that FSDV can dynamically reduce indirection mapping table space with only small performance overhead. We also demonstrate that with our design of FSDV, the changes needed in file system, flash devices, and device interface are small."

sp332 12y ago

That sounds fine in principle, but please don't tell me the future is jffs2!

wmf 12y ago
I think the proposal is something like ZFS/Btrfs but with even more complexity to handle wear leveling and such.
- etcet 12y ago
  Is this going to have a net benefit? To my mind it makes more sense for specialized functionality to be built into the drives firmware rather than letting the CPU deal with it.

Someone1234 12y ago

The title is odd. Should be "Why SATA is now obsolete." The article is of course correct, and some manufacturers are already offering SSD-like storage which is connected like a stick of RAM. I'm sure the gap between storage and the main system pipeline (CPU-GPU-RAM-etc) will only shrink as time moves forward. However as an interim solution things that acted like hard drives were convenient.

hga 12y ago
But there are also SAS (Serial Attached SCSI) SSDs ... the major point is in the aphorism in his third paragraph: "All problems in computer science can be solved by another level of indirection, except for the problem of too many layers of indirection." Hence this level of indirection, the discrete "drive", is what he thinks should go poof.
- jobu 12y ago
  > "All problems in computer science can be solved by another level of indirection, except for the problem of too many layers of indirection." First time I've heard that quote, but I love it. Reminds me of this one: "There are only two hard problems in Computer Science: cache invalidation, naming things, and off-by-one errors."
  - Already__Taken 12y ago
    These are way better than that bloody binary joke in every computer department.
    - leeoniya 12y ago
      the better version of the binary joke is Binary: as easy as 1, 10, 11
      - Someone1234 12y ago
        As easy as 3, 2, 1? I realise you read binary from right to left, not left to right, but the sentence is in English so you read that left to right. Also the commas and binary mixed together are a little odd. Least of all because you didn't make all three two binary digits (i.e. 11, 10, 01). Regardless that joke is going to cause more arguments than laughs.

blt 12y ago

Linkbait title is linkbait, but content is sound.

jkot 12y ago

> The obvious question is: why maintain two logical address spaces? There are perhaps 10 more address spaces and translations. It the way OS and hardware are build. > Why not have the file system directly manage the flash? Too slow innovation, it takes decade to replace operating system. Also main CPU is very power hungry and slow... More likely the SSD controller will become part of CPU and SOC.

KaiserPro 12y ago

"why maintain two logical address spaces? Why not have the file system directly manage the flash?" Because managing flash is hard. For this to work, the amount of exceptions in the filesystem it's self would make it unmanageable. an example is this: a Fusion-io card up until recently used low quality flash, yet managed to get stella performance. Much higher performance than the equivalent intel ssd card (who have access to much higher quality flash) some of that is down to layout, the rest firmware. Then you have the flash that lives on a device like your phone, however the firmware of the SoC is tweaked to run that particular flash controller. At its heart, flash/hdd is just a key:value database (presented with 4k value size) why would we want to complicate that (sure we need to indicate to the hardware that we've deleted a page, but to be honest not much else)

gcb0 12y ago
same was told about cylinders and sectors a couple decades ago. to use your example: imagine if you could use the fusion-io FS on the intel flash. also remember that several SSD companies that even had their own flash fab are now gone because their firmware was crap.
rasz_pl 12y ago
>Then you have the flash that lives on a device like your phone, however the firmware of the SoC is tweaked to run nowadays all of the things (ha) use flash behind higher level of abstraction, be it emmc, UFS, or SD controller. There are no phone soc firmware tweaks.
stcredzero 12y ago
At its heart, flash/hdd is just a key:value database (presented with 4k value size) why would we want to complicate that To avoid an inherent bottleneck?. Perhaps we'd be better served by a larger number of key:value stores with greater parallelism?
- wmf 12y ago
  SSDs are already parallel and I don't think updating the single FTL mapping table is a bottleneck.

stcredzero 12y ago

now that non-volatile memory technology - flash today, plus RRAM tomorrow - has been widely accepted, it is time to build systems that use flash directly instead of through our antique storage stacks. Hear hear! The basic storage model that arose in the 50's and 60's being the best fit for recently developed hardware seems as likely as the security models from the 50's and 60's being the best fit for the Internet. (If the obnoxious popup adverts from ZDNet strike me as archaic, they are not doing well.)

jschwartzi 12y ago
If NAND continues to function the way it does today, we would still need to standardize on some kind of interface to get traction. The advantage of SATA, SAS, PCIe, et. al. are their pervasiveness in the PC world. His proposal basically boils down to "NAND on SATA is kinda slow because of all the latency from stuff. We should just hook the NAND up to one of the upstream buses." It's unclear to me how this is different from what is already being done.

Animats 12y ago

The author is on to something. Flash memory devices can emulate disks, but that's an inefficient way to access them, especially for reading. You have all the overhead of Linux system calls and file systems to, basically, read a RAM-like device that's orders of magnitude faster in access time than a rotating device. The question is, what non-file abstraction should a flash device offer? Slow persistent RAM? A key/value store? An SQL database? Computing has had "RAM" and "disks" as its storage models for so long that those concepts are nailed into software at a very low level. The NoSQL crowd could probably find uses for devices that implemented big, fast key/value stores.

wmf 12y ago
Some of my coworkers bypassed the OS for Redis, although the FTL is still there. http://public.dhe.ibm.com/common/ssi/ecm/po/en/pos03135usen/POS03135USEN.PDF http://public.dhe.ibm.com/common/ssi/ecm/po/en/pos03135usen/...
- sanderjd 12y ago
  Funny how your comment told me more about what that offering is supposed to do than the whole Solution Brief did! All the same, very neat stuff.
dfox 12y ago
The problem with NAND flash is that it is not "RAM-like device" even for reads (smallest addressable entity is block even for reads). It can essentially do three operations: read block, write block, erase bunch of blocks at once. For the purposes of filesystem, this is actually an workable storage interface that does not need any additional abstraction layers.
- robotresearcher 12y ago
  You are both right. It's a RAM-like key-value store with large (block-sized) values.
  - 12y ago

danbruc 12y ago

One word. Separation of concerns. Okay, three. Why would you want to mix managing files and transistors?

jwatte 12y ago

Yes, the SATA interface to flash storage is obsolete. Just like internal combustion engines! And it will be almost as hard to get rid of.

wtallis 12y ago
SATA and AHCI are already on their way out, being replaced by PCIe and NVMe. That removes most of the abstraction that is actually obsolete, but it doesn't pretend that SSDs aren't fundamentally block devices. About the only thing NVMe does that is perpetuating a fiction about the underlying flash is that the read block size and write block size are the same thing. All the rest is abstraction that's absolutely necessary until we completely change our conventions for computer architecture and operating systems.

mark-r 12y ago

I've always wondered how much performance gain you could get if the very large blocks required for erasure were known to the file system, rather than being hidden behind an emulation layer.

ebbv 12y ago

It sounds like a great idea to "get rid of abstraction" and have the file system handle the NAND directly if you don't think about it. But if you think about it for even a little bit you realize that you can't just get rid of the abstraction, you're just moving it out of the drive's firmware and into the file system. And that gets you nothing because now the file system has to be compatible with every manufacturer's NAND drives. And the file system has to be updated any time that system needs to be changed. You can't just pop a new NAND drive in and have it work, you have to download new NAND drivers for the file system, etc. I'm also skeptical of just how much performance gain you'd see. The article doesn't include these figures, which makes me skeptical that they are significant. Delegation of duties is important for good system design. Drives necessarily need to present themselves in a standard way to the system. Having SSDs appear like normal drives is fine. They aren't memory and they shouldn't be used as memory.

sanxiyn 12y ago
Flash Translation Layer is critical to performance. Googling "flash translation layer performance" should be enough to convince you of this. Moving it from firmware to file system does not improve performance by itself, but it enables optimizations. And FTLs in firmware are far from optimal.

rwmj 12y ago

If you're interested in this subject from a Linux/storage expert, here's Ric Wheeler's talk from FOSDEM last year: https://archive.fosdem.org/2014/schedule/event/persistent_memory/ https://archive.fosdem.org/2014/schedule/event/persistent_me...

jordanpg 12y ago

I'm not a hardware expert, but my intuition tells me that the general thesis -- that any given hardware module could be implemented in some way that's light years ahead of where we are now -- is often true. But there is more to widespread adoption than being the best technology: marketing, timing, industrial realities, legacy software limitations... Legacy enterprise software and OSs in particular seem they probably make significant architectural changes to the very core of every computer difficult to effect.

lbenes 12y ago

As a related note, if you upgrade to a SSD be sure to change the SATA type in BIOS from IDE to AHCI. Not only did I see a doubling in write speed [1], but after making the change, my system felt much more responsive under heavy load. [1] http://forum.crucial.com/t5/Crucial-SSDs/Why-do-i-need-AHCI-with-a-SSD-Drive-Guide-Here-Crucial-AHCI-vs/td-p/57078 http://forum.crucial.com/t5/Crucial-SSDs/Why-do-i-need-AHCI-...

gumby 12y ago

Basically the author is calling for a return to the Multics file system, where files were just a way of referring to capability-managed subsets of an enormous pool of pages. I liked the idea the first time around so I like it again. But it's harder than it sounds. I find something lovely and poetic in the fact that Lisp is over 50 years old and Multics is almost that old, and both may be having a bit of a renaissance.

rocky1138 12y ago

This isn't an entirely novel idea. I remember in 2009 Steve Wozniak giving a talk in Waterloo about becoming the Chief Scientist for Fusion-io (https://fusionio.com/ https://fusionio.com/), a company building PCI-E SSDs, bypassing the SATA bus.

pyromine 12y ago
This is somewhat tangential, but my professor recruited Steve to come to Fusion-io and always speaks incredibly highly of Steve. When he was leaving the CEO position to give it back to the founder of Fusion-io he offered Steve some additional shares and Steve told him to give the shares the engineers who actually produce the technology. Incredible humbleness there.
pyromine 12y ago
This is somewhat tangential, but my professor recruited Steve to come to Fusion-io and always speaks incredibly highly of Steve. When he was leaving the CEO position to give it back to the founder of Fusion-io he offered Steve some additional shares and Steve told him to give the shares the engineers who actually produce the technology. Incredible humbleness there.
pyromine 12y ago
This is somewhat tangential, but my professor recruited Steve to come to Fusion-io and always speaks incredibly highly of Steve. When he was leaving the CEO position to give it back to the founder of Fusion-io he offered Steve some additional shares and Steve told him to give the shares the engineers who actually produce the technology. Incredible humbleness there.

WaxProlix 12y ago

Anyone else think it's a little ironic to read about something being obsolete on ZDNet?

markbnj 12y ago

I think the author misunderstands the word "obsolete." Something is only obsolete when there is something better to replace it with.

duckingtest 12y ago

My SSD has a three core cpu dedicated to it. Removing the layer of indirection isn't going to make it faster. At best it's going to make the ssd cheaper by a minuscule amount due to a simpler controller.

wmf 12y ago
One core of your main processor is faster than those three embedded cores; that's why Fusion-io mooches off the main processor.
- duckingtest 12y ago
  Its probably fast enough for its purpose. If it isn't, making it faster is way easier than changing existing abstraction layers. In the extreme, it can ever be an asic.

markhahn 12y ago

the basic premise is that remapping is necessarily expensive. unfortunately, neither the thesis nor its citations really support that. and we have plenty of examples where remapping is pervasive and apparently efficient (MMUs, for instance). it's hard to imagine how a filesystem could reasonably handle wear-levelling, not to mention the now-common practice of using MLC cells to hold SLC data for faster writes, etc. the "nameless writes" idea is OK - a FS person would just call it even lazier allocation, not anything really different.

zobzu 12y ago

all he means behind the sensationalistic crap is that the sata controller is not actually needed, its there for compatibility. "hard disk" will not be the ram. its just is made of the same chips, but still logically separated. and there is no need for a sata controller to address it really. That's all. You still need a file system if you want to address the files. Just like address blocks of ram or anything else. If you give free reign to the app, there is no security, reliability, etc. Even ram is actually indexed as well. On top of that you need a representation for the human behind it - no matter how hard they try to remove that (because its much easier to lock you in when there is no generic way to browser a memory device), its not very convenient compared to a file system.

deleted 12y ago

[deleted]