6 ms·
I think you're misreading this. Nobody will just bolt a few thousand of these drives to a switch and call it done. That's not practical. On the other hand it o
by astrodust 13y ago
I think you're misreading this. Nobody will just bolt a few thousand of these drives to a switch and call it done. That's not practical.
On the other hand it opens up a massive opportunity for people to create their own storage fabric using these drives as building blocks. That means instead of having to hit up EMC for a big drive array, you will eventually be able to get an open-source implementation of same if you want, or one built on open standards so you're not locked into a particular vendor.
For companies like Facebook, Google, Apple or Yahoo that are storing petabytes of information, a drive subsystem like this is surely a dream for their engineers. Now instead of having to attach the drives to servers that do little more than wrap S-ATA or SAS into Ethernet for merging into a larger storage cluster, the drive does that all by itself.
Plus, imagine how Backblaze might be able to re-engineer their pod (http://blog.backblaze.com/2013/02/20/180tb-of-good-vibrations-storage-pod-3-0/ http://blog.backblaze.com/2013/02/20/180tb-of-good-vibration...) to use this.
- nathanb 13y agoI think you're missing the value prop of the BigStorageCos like EMC and NetApp. If all they provided was a disk shelf and loose cluster coupling, nobody would buy that. The big win (for those companies which consider it a win) is in data management and namespacing. Storage virtualization, if you will. I can have a hundred disks spinning behind a storage controller which lumps them together into three storage volumes. I can serve file data off two of them (say, NFS on one and CIFS on the other) and stick some LUNs on the third and serve blocks (iSCSI). I can dedupe, snapshot, and migrate these data at will. FWIW. Not belittling the Seagate announcement. Just clarifying why the article is correct in suggesting that EMC and NetApp aren't particularly worried about the announcement.
- astrodust 13y agoObviously EMC, NetApp and others in that space provide value, but what they do with existing drive technology is simply out of reach of most startups. The barrier to entry is way too high. With this new technology it opens the door for innovation in this space. Using commodity components you can build out new EMC-type systems with a different focus. The amount of hardware engineering you have to do is minimized, and more of it can be done strictly in software. Lower cost, more rapid iteration, all that. I'm not saying this will happen tomorrow, but in ten years the storage market could be turned inside out by this approach.
- peterwwillis 13y agoI am imagining it. The only effectively useful feature of this is actually adding an extra server layer, not removing one. -- Case 1: Startup X makes a webapp cluster that looks up user information and returns results. It calls a library, which looks up a hash key to query a disk, and returns data. Problem 1: Lack of load balancing. If there are three disks, and user FRANK is on disk two, and user FRANK's data is getting queried 50x more than the other users, that second disk is toast performance-wise. Problem 2: No redundancy plus short lifespan of disk means when the disks die the user data goes too. -- Case 2: Big Company Y creates a storage application layer to intelligently do things with the data. They have a small cluster of machines with apps that take queries and do things with the data, and manage the data using key/value pairs on disks attached to a private storage switch. Problem 1: Dependent on ethernet (and its overhead, and latency) for each query doesn't perform as fast as other disk interconnects; have to use hacks to increase performance. Network management now critical component of your storage functionality. Problem 2: Because the Virtual Memory Manager is no longer managing a filesystem cache, all key/value fields must be cached by the application, so you're re-implementing a VMM layer in your storage application. (Because nobody is stupid enough to not cache random disk queries) Problem 3: Relational queries become almost completely useless. Performance drags due to all the individual queries, and you end up building a new cache layer just so your database can speed up searches, or at worst case end up with an index-only cluster of these disks. -- As you can tell by reading Backblaze's site, there are lots of different uses for storage and different requirements for each. But one thing that's pretty widely acknowledged is it's more efficient to have a really long single piece of storage versus lots of very short pieces. I imagine Backblaze will look at this and go: Why don't we just make our own?