6 ms·
I thought RAID and SSD were a terrible combination. As if both drives do exactly the same writes at the same time, chances are they will fail at the same time.
by cm2187 11y ago
I thought RAID and SSD were a terrible combination. As if both drives do exactly the same writes at the same time, chances are they will fail at the same time. Plus most RAID controllers don't support TRIM.
Has the thinking changed?
- zanny 11y agoA Raid 5 could work, since you are not writing the same data. You would need a hardware coprocessor to handle generating the checksums at 4GB/s though.
- pixl97 11y ago>You would need a hardware coprocessor to handle generating the checksums at 4GB/s though. A first gen i5 2.4GHz can generate XOR for RAID5 at 4GB/s. Newer generations at faster core speeds should be able to beat that significantly. ... I went and tested this on a Intel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz [1053854.445872] avx : 17860.000 MB/sec [1053854.475878] raid6: sse2x1 gen() 6554 MB/s [1053854.492857] raid6: sse2x2 gen() 9070 MB/s [1053854.509853] raid6: sse2x4 gen() 11273 MB/s [1053854.526848] raid6: avx2x1 gen() 21554 MB/s [1053854.543844] raid6: avx2x2 gen() 25152 MB/s [1053854.560838] raid6: avx2x4 gen() 29167 MB/s [1053854.560839] raid6: using algorithm avx2x4 gen() (29167 MB/s) [1053854.560840] raid6: using avx2x2 recovery algorithm
- ryao 11y agoRAID 5 is terrible with 4K random IO. I know guys who measured >1 million IOPS with MD RAID 0 and then switched to MD RAID 5 only to have performance drop from read-modify-write overhead. They could not believe that RAID 5 was the cause.
- chipperyman573 11y agoYeah, the statistical chance of the same thing happening on both drives is the same, but the chance of it happening at the same time is pretty low.
- stray 11y agoThat sounds perfectly reasonable, and 20 years ago I'd have said the same thing. It's even fairly likely that I did say it. You know where this is going, right? I was a SysAdmin at the time and one of the things I was responsible for was Oracle Financials. It was at the core of everything that mattered at that company -- and I was absolutely convinced that short of a water balloon fight in the machine room, my raid configuration was 100% reliable. The first disk died late on Friday. I had two on-hand so I casually replaced it -- and before I got back to my desk another had died. I placed an order for more drives and went back to the machine room and replaced it. By Monday morning, things were starting to get serious -- I had to drop back to raid 5 because a few more had died over the weekend and my replacements wouldn't arrive till Tuesday. You see, all the drives in all our raid enclosures had come from the same batch -- and they all -- every single one of them -- died within 90 days of the first one. The chances may have been low, but reality has sharp teeth and loves the taste of overconfident sysadmin ass.
- ars 11y ago> The first disk died late on Friday. I had two on-hand so I casually replaced it -- and before I got back to my desk another had died. That happens very very very often. It has nothing to do with a bad batch. The second disk actually failed a while ago, but no one noticed because no one read from that part of it. When you did the rebuild you read from the failed area and woke up the failure. When you setup raid you MUST read the entire raid at least monthly, so that any errors are detected early! This is absolutely critical. Without that you have not installed the raid correctly. mdadm on debian does that by default. Linux has a builtin way to do that, but you must have a tool that will alert you on failure, or it's worthless. You should also run a week full disk read of each hard disk in the array using its onboard long self test feature. You can use smartd to schedule that automatically. And more importantly: Notify you on failure. Not using both tools is not setting up the raid correctly.
- cookiemonsta 11y agoIs there a well known tool to check for this?
- sdesol 11y ago> I thought RAID and SSD were a terrible combination Well there are different RAID settings. My product is an indexing engine (indexes Git repositories) and I've personally found RAID 0 reduces indexing time by about 30% compared to a single drive. I have a machine that has 4 SSDs running RAID 0 and I've found the performance gain after 3 SSDs is negligible. It seems like 3 SSDs running RAID 0 is the best combination given my very limited sampling size.
- laurencerowe 11y agoAn SSD is implemented a little like a RAID array internally. Higher capacity SSDs have more flash chips and tend to be faster. So one would expect that 2 x 128GB drives in RAID 0 to perform similarly to a single 256GB drive.
- deleted 11y ago[deleted]
- deleted 11y ago[deleted]
- gfody 11y agoSoftware RAID or a host-based controller is the way to go with SSDs. Otherwise the raid controller becomes your bottleneck as whatever onboard ASIC or RAID-on-a-chip hits its throughput limit (and potentially overheats and dies if it's some crappy product designed for HDDs). TRIM is well supported by modern host-based controllers designed for SSDs.
- Sanddancer 11y agoThey'll have the same lifespan, but they won't fail at exactly the same time. Also, a lot of RAID now is done in software, and most software raid types support TRIM.
- jlgaddis 11y agoI just (like, about the time you were writing your comment) replaced a pair of spinning disks with a pair of 120 GB SSDs in one of my machines here at home. It's not technically RAID but a ZFS mirror instead. The operating system and related files will live on it while my data still lives on some spinning disks (simply because of the amount of data). I've got pretty much the same setup (albeit with several more spinning disks) in a bunch of servers and have yet to have any problems (* crosses fingers *).
- Freaky 11y ago> As if both drives do exactly the same writes at the same time, chances are they will fail at the same time. You mean due to exhausting their endurance? This is something you monitor, you should have plenty of time to replace the drives before it becomes a concern. For other failures, how's it going to be any different from normal HD's? There's always the risk that having the same models/batches in the same conditions might lead to a cluster of failures, but RAID's still likely to save you from plenty of other failure modes. For instance, the recent Google study[1] suggests that uncorrectable errors with SSDs are actually more common than HD's: "More than 20% of flash drives develop uncorrectable errors in a four year period... significantly higher rates .. than hard disk drives". I've certainly seen enough IO errors from SSDs to know I want to defend against them. Not to mention the silent data corruption I've seen (and been protected from, yay ZFS). 1: http://0b4af6cdc2f0c5998459-c0245c5c937c5dedcca3f1764ecc9b2f.r43.cf2.rackcdn.com/23105-fast16-papers-schroeder.pdf http://0b4af6cdc2f0c5998459-c0245c5c937c5dedcca3f1764ecc9b2f...
- rsync 11y ago"You mean due to exhausting their endurance? This is something you monitor, you should have plenty of time to replace the drives before it becomes a concern. For other failures, how's it going to be any different from normal HD's? There's always the risk that having the same models/batches in the same conditions might lead to a cluster of failures, but RAID's still likely to save you from plenty of other failure modes." He's talking about something else. Physical drives fail due to physical reasons, but SSDs fail for logical reasons. A weird firmware bug or an unexpected usage pattern or wear pattern (not just burning the entire things lifetime up) can cause the SSD to die. If you have a mirror, and you have two identical SSDs, and you produce such a condition ... then no more mirror. They both die identically. See my response to the parent about how you can easily address this by mixing SSDs in mirrors.
- cm2187 11y agoI am actually referring to endurance. Irrespective of the RAID mode, I would expect each drive to be written exactly the same amount of data at the same time. If all models are identical and have been purchased at the same time, one would expect that the same firmware will allocate the same writes to the same cells (in RAID 1 it will be the same data, in RAID 5 the data will be slightly different, but the size will be the same). So in theory the wear should be identical. Obviously this only applies to a RAID mode where you have parity. But I read that like 3-4y ago. We should have more experience with SSDs in production now so I was wondering if that thinking still applied.
- rsync 11y ago"I thought RAID and SSD were a terrible combination. As if both drives do exactly the same writes at the same time, chances are they will fail at the same time." This is a very good thought and you should be thinking it. However, it is really only applicable to raid mirrors. Raid stripes will not have identical wear lifespans. All of our mirrored boot devices are either: - current intel enterprise SSD paired with the previous generation intel enterprise SSD or: - current intel enterprise SSD paired with current samsung enterprise SSD That way if there is a usage related failure (or some weird firmware bug triggered by use pattern) they don't seize up identically.