4 ms·

So now we will have yet another computer IN THE SSD too? Is this a good idea? https://boingboing.net/2018/11/12/cpus-in-your-cpus.html https://boingboing.net/20

So now we will have yet another computer IN THE SSD too? Is this a good idea? https://boingboing.net/2018/11/12/cpus-in-your-cpus.html https://boingboing.net/2018/11/12/cpus-in-your-cpus.html

dbalan 7y ago
It won't be much different than what the current generation storage devices contain. You already need to upgrade SSD firmware these days. Linux fwupmgr has support for few of them.
writepub 7y ago
There is already a computer in the SSD, replete with DRAM, multcore CPU, custom math acceleration (depending on SSD design and vendor), etc.
- berbec 7y ago
  The HP EX920 has a dual-core Arm Cortex.[1] The Samsung 970 Pro has the "Samsung Phoenix NVMe controller". 1: https://www.tomshardware.com/reviews/hp-ex920-ssd,5527.html https://www.tomshardware.com/reviews/hp-ex920-ssd,5527.html
duskwuff 7y ago
This has been the case for a while already. Even spinning-platter hard disks had some reasonably powerful CPUs to implement caching, bad-block remapping, and SMART data retention (among other features); SSDs pushed this to another level by demanding higher performance and more complex remapping. Chances are that this Samsung device isn't even using different hardware from their standard SSDs -- just different firmware. [1]: https://spritesmods.com/?art=hddhack&page=1 https://spritesmods.com/?art=hddhack&page=1
rhinoceraptor 7y ago
I try not to think about the fact that horrible proprietary firmware is in everything now.
wtallis 7y ago
The most active area of R&D in the storage industry at the moment is stuff falling under the umbrella term "computational storage". Everyone is looking into ways to make SSDs smarter, and move some processing closer to the storage where it can be done more efficiently or at least free up CPU time. Some of the companies making enterprise SSD controller ASICs have decided to throw in extra ARM Cortex-A53 or similar cores for the customer to put software on, giving the application access to the data without a PCIe bottleneck. Some companies are putting machine learning accelerators on the SSD. Some are adding dedicated compression or crypto engines to transform data at wire speed. Some are just putting an FPGA on the drive, or implementing the controller on an FPGA and leaving leftover LUTs for the customer's use. Almost any idea you can come up with about how to move storage hardware past the hard drive-like block storage paradigm has been at least prototyped and demoed at Flash Memory Summit.
- DonHopkins 7y ago
  It's like NFS 3 (NeFS) in hardware! (NeWS for disks: A PostScript interpreter in the kernel as a file system API.) https://www.donhopkins.com/home/nfs3_0.pdf https://www.donhopkins.com/home/nfs3_0.pdf Network Extensible File System Protocol Specification (2/12/90) Comments to: sun!nfs3 nfs3@SUN.COM Sun Microsystems, Inc. 2550 Garcia Ave. Mountain View, CA 94043 1.0 Introduction The Network Extensible File System protocol (NeFS) provides transparent remote access to shared file systems over networks. The NeFS protocol is designed to be machine, operating system, network architecture, and transport protocol independent. This document is the draft specification for the protocol. It will remain in draft form during a period of public review. Italicized comments in the document are intended to present the rationale behind elements of the design and to raise questions where there are doubts. Comments and suggestions on this draft specification are most welcome. [...] Although it has features in common with NFS, NeFS is a radical departure from NFS. The NFS protocol is built according to a Remote Procedure Call model (RPC) where filesystem operations are mapped across the network as remote procedure calls. The NeFS protocol abandons this model in favor of an interpretive model in which the filesystem operations become operators in an interpreted language. Clients send their requests to the server as programs to be interpreted. Execution of the request by the server’s interpreter results in the filesystem operations being invoked and results returned to the client. Using the interpretive model, filesystem operations can be defined more simply. Clients can build arbitrarily complex requests from these simple operations.
Tuna-Fish 7y ago
Your current SSD already has a beefy computer with lots of ram to handle the flash translation layer. Raw flash has very onerous constraints on how it can be written: (numbers are from recent QLC chips) The smallest unit that can be accessed is the individual page (64kB). Those pages belong to blocks (18MB). Each page can be either dirty or clean. You can read any page, but you can only write to clean pages (turning them dirty). To turn dirty pages back to clean, you have to erase them, and you can only erase the entire block containing the page at once. Also, each block can only be erased a certain amount of times (500) until it can no longer be written to. As you can see, the API this provides is just awful, especially considering what kind of operations (random read/write of 4kB) operating systems offer to programs. So, in order to provide an api operating systems can actually use, there is a massive garbage-collected translation layer on top of the raw flash that makes it usable. The idea of key-value SSDs is that given that the block layer api is entirely artificial, it might not be the best choice to present to the OS.

So now we will have yet another computer IN THE SSD too? Is this a good idea? https://boingboing.net/2018/11/12/cpus-in-your-cpus.html https://boingboing.net/2018/11/12/cpus-in-your-cpus.html

dbalan 7y ago

It won't be much different than what the current generation storage devices contain. You already need to upgrade SSD firmware these days. Linux fwupmgr has support for few of them.

writepub 7y ago

There is already a computer in the SSD, replete with DRAM, multcore CPU, custom math acceleration (depending on SSD design and vendor), etc.

berbec 7y ago
The HP EX920 has a dual-core Arm Cortex.[1] The Samsung 970 Pro has the "Samsung Phoenix NVMe controller". 1: https://www.tomshardware.com/reviews/hp-ex920-ssd,5527.html https://www.tomshardware.com/reviews/hp-ex920-ssd,5527.html

duskwuff 7y ago

This has been the case for a while already. Even spinning-platter hard disks had some reasonably powerful CPUs to implement caching, bad-block remapping, and SMART data retention (among other features); SSDs pushed this to another level by demanding higher performance and more complex remapping. Chances are that this Samsung device isn't even using different hardware from their standard SSDs -- just different firmware. [1]: https://spritesmods.com/?art=hddhack&page=1 https://spritesmods.com/?art=hddhack&page=1

rhinoceraptor 7y ago

I try not to think about the fact that horrible proprietary firmware is in everything now.

wtallis 7y ago

The most active area of R&D in the storage industry at the moment is stuff falling under the umbrella term "computational storage". Everyone is looking into ways to make SSDs smarter, and move some processing closer to the storage where it can be done more efficiently or at least free up CPU time. Some of the companies making enterprise SSD controller ASICs have decided to throw in extra ARM Cortex-A53 or similar cores for the customer to put software on, giving the application access to the data without a PCIe bottleneck. Some companies are putting machine learning accelerators on the SSD. Some are adding dedicated compression or crypto engines to transform data at wire speed. Some are just putting an FPGA on the drive, or implementing the controller on an FPGA and leaving leftover LUTs for the customer's use. Almost any idea you can come up with about how to move storage hardware past the hard drive-like block storage paradigm has been at least prototyped and demoed at Flash Memory Summit.

DonHopkins 7y ago
It's like NFS 3 (NeFS) in hardware! (NeWS for disks: A PostScript interpreter in the kernel as a file system API.) https://www.donhopkins.com/home/nfs3_0.pdf https://www.donhopkins.com/home/nfs3_0.pdf Network Extensible File System Protocol Specification (2/12/90) Comments to: sun!nfs3 nfs3@SUN.COM Sun Microsystems, Inc. 2550 Garcia Ave. Mountain View, CA 94043 1.0 Introduction The Network Extensible File System protocol (NeFS) provides transparent remote access to shared file systems over networks. The NeFS protocol is designed to be machine, operating system, network architecture, and transport protocol independent. This document is the draft specification for the protocol. It will remain in draft form during a period of public review. Italicized comments in the document are intended to present the rationale behind elements of the design and to raise questions where there are doubts. Comments and suggestions on this draft specification are most welcome. [...] Although it has features in common with NFS, NeFS is a radical departure from NFS. The NFS protocol is built according to a Remote Procedure Call model (RPC) where filesystem operations are mapped across the network as remote procedure calls. The NeFS protocol abandons this model in favor of an interpretive model in which the filesystem operations become operators in an interpreted language. Clients send their requests to the server as programs to be interpreted. Execution of the request by the server’s interpreter results in the filesystem operations being invoked and results returned to the client. Using the interpretive model, filesystem operations can be defined more simply. Clients can build arbitrarily complex requests from these simple operations.

Tuna-Fish 7y ago

Your current SSD already has a beefy computer with lots of ram to handle the flash translation layer. Raw flash has very onerous constraints on how it can be written: (numbers are from recent QLC chips) The smallest unit that can be accessed is the individual page (64kB). Those pages belong to blocks (18MB). Each page can be either dirty or clean. You can read any page, but you can only write to clean pages (turning them dirty). To turn dirty pages back to clean, you have to erase them, and you can only erase the entire block containing the page at once. Also, each block can only be erased a certain amount of times (500) until it can no longer be written to. As you can see, the API this provides is just awful, especially considering what kind of operations (random read/write of 4kB) operating systems offer to programs. So, in order to provide an api operating systems can actually use, there is a massive garbage-collected translation layer on top of the raw flash that makes it usable. The idea of key-value SSDs is that given that the block layer api is entirely artificial, it might not be the best choice to present to the OS.