4 ms·
If you use a fully distributed storage system, like Ceph (or 3Par?), then rebuild times during a drive failure would be significantly lower. Also, any self-res
by antongribok 11y ago
If you use a fully distributed storage system, like Ceph (or 3Par?), then rebuild times during a drive failure would be significantly lower. Also, any self-respecting storage system (RAID or not) will lower rebuild priority over normal IO.
To give a specific example, let's say you have a 10-node Ceph cluster with 10 of these SSDs in each node filled up to 75% capacity. During a rebuild you throttle down to 50 MBps per SSD. After a single drive failure, the remaining 99 SSDs will work together to redistribute the data in about 40 minutes:
$ python3 -c "print(round((15.36 * 1024 * 1024 *.75) / (50.0 * 99) / 60.0, 2))"
40.67
The 50 MBps per drive may seem like a low number, but that actually means each node has to move data at roughly 5 Gbps over the cluster network.
* Edited to fix the python line where the multiplication characters converted to italics formatting.
- beagle3 11y agoIsn't 50MBps closer to 0.5Gbps than 5Gbps? Where does the other factor of ten come from?