4 ms·
> Can it be as simple as a spanning partition to treat all drives as a single drive? Sure, but there are tradeoffs. One option is to just concatenate all the d
by teraflop 11y ago
> Can it be as simple as a spanning partition to treat all drives as a single drive?
Sure, but there are tradeoffs. One option is to just concatenate all the drives together, either with mdadm (grouping the drives into a RAID-0 array) or with LVM (adding them all to a single volume group). This lets you add drives very easily and efficiently, but has no redundancy. When (not if) one drive fails, your entire FS is toast.
If you care about your data, then you'll want to use RAID-6 for redundancy. But adding a drive to a RAID-6 array is very time-consuming, because all of your data has to be re-shuffled onto a new drive. And you don't want to do this without a backup -- although the reshape process is designed to be safely interruptible, it's not bug-free. (I speak from experience, having nearly lost a few TB this way not too long ago.)
You can compromise by grouping your drives into multiple RAID-6 arrays. For example, instead of using 58 data drives + 2 parity drives, you could use 6 arrays of (8 data + 2 parity). This gives you much less usable space (80% as opposed to 97%) but it allows you to add 10 drives at a time without having to re-shuffle all your data. And it means you can recover from a drive failure much more quickly.
Of course, you can also just treat the drives as separate block devices, and provide a single redundant filesystem at a higher level, using something like Ceph. I would assume Backblaze is doing something along these lines.
- zzzcpan 11y ago> I would assume Backblaze is doing something along these lines. They are using a regular filesystem on each drive and implement redundancy on top of regular files. It's more like Swift [1] than Ceph. Which is something parent should consider too, if he's going to deal with that much data. [1] http://docs.openstack.org/developer/swift/overview_architecture.html http://docs.openstack.org/developer/swift/overview_architect...
- bwilliams18 11y agoHas anyone built a Swift implementation in Swift?
- AdamJacobMuller 11y agoSomeone should if they haven't already. But seriously I do hate namespace collisions like this. Swift (the storage architecture) is something i've been a fan of for years, swift (the programming language) i've dabbled in a bit and its interesting for some use cases for sure (aside from iOS programming which I'm not interested in at all). The cognitive dissonance of switching back-and-forth is annoying. I do hope nobody ever writes or needs to use a swift API connector in swift, that would be painful.