4 ms·

I have avoided RAID for the past 20 years and will avoid it in future. The whole technology and its benefits is about picking pennies before the steamroller, wi

I have avoided RAID for the past 20 years and will avoid it in future. The whole technology and its benefits is about picking pennies before the steamroller, with explosive complexity of failures. I prefer much more simple and understandable replication and backup solutions - there are dozens of them.

throwaway-9320 7y ago
Have you tried out ZFS? It's quite good at resolving failures and data corruption and can be configured in all sorts of combinations, depending on your performance and disk failure tolerance requirements.
- juskrey 7y ago
  Yes, it is good!
bouncycastle 7y ago
It's very important to understand that RAID is not a backup solution. You still need to backup even if you have RAID, no matter what level. What it offers (in parity mode) is improved reliability and more chance that your server will be online if your drive dies. In majority of cases, it saves you from restoring from backup. Just replace the bad drive, and away you go! However, never assume that it will protect you from data loss. Backup. Backup. Backup!
- MaxBarraclough 7y ago
  Also, test your backups regularly.
- prussian 7y ago
  Yes, but what the parent comment is saying is that simple replication can replace the need for parity arrays entirely.
  - juskrey 7y ago
    Exactly. One doesn't have to be superhuman to set up modern DB replication and manage its failure modes. Even if you need to salvage broken HDD, it's a pretty straightforward task with high rate of success.
  - Rychard 7y ago
    In the sense that your backups are a surviving copy of your data, sure. That said, an argument such as "system is down while we restore from backup; some recent data may be lost" will not go over as well as "the system remains available, but performance will be negatively impacted while the array rebuilds; no data has been lost at this time". In the former, you're in crisis mode. In the latter, you'll probably be under some stress, but you're still in the clear. Edit: But we're all on the same page when it comes to backups. They're a necessity.
    - 411111111111111 7y ago
      I'm pretty sure he's thinking more about distributed filesystems vs raids than what you're talking about. I.e. ceph with pods distributed over a bunch of disks without a raid
  - SlowRobotAhead 7y ago
    > that simple replication can replace the need for parity arrays entirely. Not if you need speed it doesn’t.
- PaulHoule 7y ago
  Trouble is that the industry has not kept up with people's backup needs. A tape drive costs ten times what a hard drive with the same capacity costs, and then you have to buy the tapes. I've been able to read data written to a hard drive 99.99% of the time but I can't say that for tapes. Often it has been human error, but the fact that it's a strange thing you don't do everyday and isn't part of your workflow is a reliability problem that is hard to overcome. (e.g. why a Toyota Corolla is more reliable than a Ferrarri Testarosa.) In theory you could do offline backup to "the cloud" but probably your net isn't fast enough and the other people at home will be wondering why Zoom isn't working for them.
- AstralStorm 7y ago
  Technically, disconnected RAID1 (or RAID10 multiple) is a backup. Recovery is a bit funny with you marking all remaining drives as faulty then plugging backup for resilvering. (Or you can use it directly, less safely, as a normal separate array.) Must be disconnected and unpowered, yet regularly checked. Not super useful for extreme long term storage but neither are tapes.
  - zepearl 7y ago
    > Technically, disconnected RAID1 (or RAID10 multiple) is a backup Mmhh, sounds weird - what do you mean with "disconnected RAID1"?
  - bouncycastle 7y ago
    Not familiar with the "disconnected" technique - does this mean you disconnect & reconnect your drive periodically? Sounds like a misuse of RAID features. What could possibly go wong? Eg. Reconnect the drive accidentally forgetting to mark others as bad and bye bye "backup"? Also, RAID controllers die sometimes. It's not pretty when that happens, which means you really need a backup of the whole array anyways. So technically RAID is not a backup because you are still not protected from that type of failure.
deleted 7y ago
[deleted]
rconti 7y ago
How have you handled the "storing lots of data on a volume" problem? This seems a bit like someone saying "my blog never needed Kubernetes". Of course not, that's not what it's for.
- juskrey 7y ago
  I have divided the data onto smaller volumes.
  - rconti 7y ago
    Right, that's my point. Not really feasible for 300TB databases or 50TB vmfs datastores. Obviously those things can be worked around with complete re-architecting of, well, everything, but I'm considering that out-of-scope.
gnulinux 7y ago
Sorry, not very experienced in RAID. But when you say RAID do you mean RAID 10 or something? Isn't "replication" (i.e. having a mirror as a backup) literally just RAID 1?
- SlowRobotAhead 7y ago
  10 is one set of drives striped in 0 for for speed, and another identical set mirroring them as 1. 10 is 1+0

throwaway-9320 7y ago

Have you tried out ZFS? It's quite good at resolving failures and data corruption and can be configured in all sorts of combinations, depending on your performance and disk failure tolerance requirements.

juskrey 7y ago
Yes, it is good!

bouncycastle 7y ago

It's very important to understand that RAID is not a backup solution. You still need to backup even if you have RAID, no matter what level. What it offers (in parity mode) is improved reliability and more chance that your server will be online if your drive dies. In majority of cases, it saves you from restoring from backup. Just replace the bad drive, and away you go! However, never assume that it will protect you from data loss. Backup. Backup. Backup!

MaxBarraclough 7y ago
Also, test your backups regularly.
prussian 7y ago
Yes, but what the parent comment is saying is that simple replication can replace the need for parity arrays entirely.
- juskrey 7y ago
  Exactly. One doesn't have to be superhuman to set up modern DB replication and manage its failure modes. Even if you need to salvage broken HDD, it's a pretty straightforward task with high rate of success.
- Rychard 7y ago
  In the sense that your backups are a surviving copy of your data, sure. That said, an argument such as "system is down while we restore from backup; some recent data may be lost" will not go over as well as "the system remains available, but performance will be negatively impacted while the array rebuilds; no data has been lost at this time". In the former, you're in crisis mode. In the latter, you'll probably be under some stress, but you're still in the clear. Edit: But we're all on the same page when it comes to backups. They're a necessity.
  - 411111111111111 7y ago
    I'm pretty sure he's thinking more about distributed filesystems vs raids than what you're talking about. I.e. ceph with pods distributed over a bunch of disks without a raid
- SlowRobotAhead 7y ago
  > that simple replication can replace the need for parity arrays entirely. Not if you need speed it doesn’t.
PaulHoule 7y ago
Trouble is that the industry has not kept up with people's backup needs. A tape drive costs ten times what a hard drive with the same capacity costs, and then you have to buy the tapes. I've been able to read data written to a hard drive 99.99% of the time but I can't say that for tapes. Often it has been human error, but the fact that it's a strange thing you don't do everyday and isn't part of your workflow is a reliability problem that is hard to overcome. (e.g. why a Toyota Corolla is more reliable than a Ferrarri Testarosa.) In theory you could do offline backup to "the cloud" but probably your net isn't fast enough and the other people at home will be wondering why Zoom isn't working for them.
AstralStorm 7y ago
Technically, disconnected RAID1 (or RAID10 multiple) is a backup. Recovery is a bit funny with you marking all remaining drives as faulty then plugging backup for resilvering. (Or you can use it directly, less safely, as a normal separate array.) Must be disconnected and unpowered, yet regularly checked. Not super useful for extreme long term storage but neither are tapes.
- zepearl 7y ago
  > Technically, disconnected RAID1 (or RAID10 multiple) is a backup Mmhh, sounds weird - what do you mean with "disconnected RAID1"?
- bouncycastle 7y ago
  Not familiar with the "disconnected" technique - does this mean you disconnect & reconnect your drive periodically? Sounds like a misuse of RAID features. What could possibly go wong? Eg. Reconnect the drive accidentally forgetting to mark others as bad and bye bye "backup"? Also, RAID controllers die sometimes. It's not pretty when that happens, which means you really need a backup of the whole array anyways. So technically RAID is not a backup because you are still not protected from that type of failure.

deleted 7y ago

[deleted]

rconti 7y ago

How have you handled the "storing lots of data on a volume" problem? This seems a bit like someone saying "my blog never needed Kubernetes". Of course not, that's not what it's for.

juskrey 7y ago
I have divided the data onto smaller volumes.
- rconti 7y ago
  Right, that's my point. Not really feasible for 300TB databases or 50TB vmfs datastores. Obviously those things can be worked around with complete re-architecting of, well, everything, but I'm considering that out-of-scope.

gnulinux 7y ago

Sorry, not very experienced in RAID. But when you say RAID do you mean RAID 10 or something? Isn't "replication" (i.e. having a mirror as a backup) literally just RAID 1?

SlowRobotAhead 7y ago
10 is one set of drives striped in 0 for for speed, and another identical set mirroring them as 1. 10 is 1+0