4 ms·

People who use filesystems like this seem to completely misunderstand what RAID is for. They seem to care about their data somewhat, which of course means they

People who use filesystems like this seem to completely misunderstand what RAID is for. They seem to care about their data somewhat, which of course means they have off-site backups. Given that they have reliable backups, redundancy is only useful to the extent that it improves uptime. Uptime is best maintained by eliminating single points of failure. Raid is a good first step, but at some point, creating a massive disk array on a single motherboard/cpu/disk-controller is anything but that. And they don't seem to be complaining of downtime. Given that they have already achieved data safety, and don't care about uptime, the only reasonable explanation I can surmise is that they're confused.

gnoway 11y ago
In your opinion, what is the best way to address the use case of spreading I/O across multiple spindles for non-uptime-related performance reasons?
- oconnore 11y ago
  Using any of the btrfs supported RAID levels will improve read performance, and anything but level 1 will improve write performance. I like Raid10.
- toomuchtodo 11y ago
  Who even cares about spindles anymore? Need fast? SSD on the PCI bus. Need lots of storage? Spinning disk where you care not about performance.
  - simoncion 11y ago
    > Who even cares about spindles anymore? Folks who have a shitload of data to store, but want more perf than a single drive can give them?
rapala 11y ago
Does the off-site backup also contain the write that happened 1.5 seconds ago? What about the corruption that happened just before taking the backup?
- deleted 11y ago
  [deleted]
- oconnore 11y ago
  This is a ridiculous scenario. If you have some files on your laptop that are super important, you probably shouldn't `rm -rf` them 1.5 seconds after uploading to your NAS. If you were working on that file, you shouldn't wait to save it until you are finished, or should be prepared to redo that work. Save it as you go. Anyway, if a drive fails 1.5 seconds after a write, any of the btrfs supported Raid levels (-0) will save you. If corruption happens just before a backup, no Raid level will save you, because Raid does not protect against undetected silent corruption. Btrfs may save you there, because it does computes checksums -- allowing you to detect the corruption and recover. If you really are going to immediately purge some critical data from its source location 1.5 seconds after fsyncing to your drive, you should be sending it to multiple locations and waiting for confirmation containing a valid checksum before considering the write a success.
  - rapala 11y ago
    > This is a ridiculous scenario. If you have some files on your laptop that are super important, you probably shouldn't `rm -rf` them 1.5 seconds after uploading to your NAS. There is nothing ridiculous about a hard drive breaking between backups. There are situations where loosing all data after latest backup would be, if not unrecoverable, at least extremely irritating. Having redundant drives in a laptop might not be feasible, but dedicated workstations are still used in many places. > If you really are going to immediately purge some critical data from its source location 1.5 seconds after fsyncing to your drive, you should be sending it to multiple locations and waiting for confirmation containing a valid checksum before considering the write a success. Which is exactly what ZFS/btrfs in a mirrored RAID configuration does.
jrockway 11y ago
I tend to agree. RAID 1 is fine for your desktop where you don't want to waste time restoring your backup when a drive inevitably dies. When the kernel gets confused and says both drives are corrupted simultaneously, it's easy to manually repair with dd. (See which block has unrecoverable errors and copy that one over from your other drive. Repaired. Not sure why the kernel can't do this automatically. The data is gone, you can't make it worse.) For datacenter applications, having everything on one machine behind a filesystem abstraction seems like unnecessary complexity, not to mention a single point of failure and a waste of resources. I'd much rather handle things on the application level; if I want three copies of my data around, write it to three different machines with disks. When a copy seems unavailable, make an extra to keep the redundancy level the same across disk failures. If I want some redundancy encoding, just store that instead of the raw file. Now there is no complicated recovery when your disk fails. The system is always in a recovered state because disk failures are expected and uneventful. That said, there is a bit of complexity in implementing this, and even more complexity when you want all the pleasantries of a UNIX filesystem (permissions, quota, etc.). I know of only one working implementation of this technique, and it's not open source. And, of course, if you're going to use existing software rather than writing your own, you're going to need a UNIX filesystem for it to use, not some clever chunk storage manager you wrote. So btrfs and zfs persist.
skorgu 11y ago
There are lots of areas in which the data is reproducible but local continuity (and checksumming!) are still extremely valuable. Think things like large iso files, dvd rips or even intermediate stages of data analysis. Since I can reproduce them having a backup might not make sense but I still don't really /want/ to have to go to all that effort. Especially things like scratch space for analysis you really don't want bit flips since you've maybe silently invalidated later stages.
- oconnore 11y ago
  Did you read the article? Did you read my post? Who are you replying to? Checksumming is great. Improving uptime (in your case, by not having to recreate the data) is great. Why do you assume I don't agree with you?
  - skorgu 11y ago
    "They seem to care about their data somewhat, which of course means they have off-site backups." I don't think a reply of "there are classes of data that one can care about and still not have backups of" is unreasonable. "...I can surmise is that they're confused" Again, I think showing a few cases in which users can have perfectly non-confused reasons for using these filesystems is pretty straightforward as conversational gambits go.
    - oconnore 11y ago
      The article wasn't about using RAID, it was complaining of a lack of RAID60 and triple parity in btrfs, and a lack of volume expansion features in zfs.
      - skorgu 11y ago
        Well, yes but I don't think either of those things are really addressed by "care about data IFF backups exist". I guess I either don't get what point you're making or we're just getting distracted by phrasing.

gnoway 11y ago

In your opinion, what is the best way to address the use case of spreading I/O across multiple spindles for non-uptime-related performance reasons?

oconnore 11y ago
Using any of the btrfs supported RAID levels will improve read performance, and anything but level 1 will improve write performance. I like Raid10.
toomuchtodo 11y ago
Who even cares about spindles anymore? Need fast? SSD on the PCI bus. Need lots of storage? Spinning disk where you care not about performance.
- simoncion 11y ago
  > Who even cares about spindles anymore? Folks who have a shitload of data to store, but want more perf than a single drive can give them?

rapala 11y ago

Does the off-site backup also contain the write that happened 1.5 seconds ago? What about the corruption that happened just before taking the backup?

deleted 11y ago
[deleted]
oconnore 11y ago
This is a ridiculous scenario. If you have some files on your laptop that are super important, you probably shouldn't `rm -rf` them 1.5 seconds after uploading to your NAS. If you were working on that file, you shouldn't wait to save it until you are finished, or should be prepared to redo that work. Save it as you go. Anyway, if a drive fails 1.5 seconds after a write, any of the btrfs supported Raid levels (-0) will save you. If corruption happens just before a backup, no Raid level will save you, because Raid does not protect against undetected silent corruption. Btrfs may save you there, because it does computes checksums -- allowing you to detect the corruption and recover. If you really are going to immediately purge some critical data from its source location 1.5 seconds after fsyncing to your drive, you should be sending it to multiple locations and waiting for confirmation containing a valid checksum before considering the write a success.
- rapala 11y ago
  > This is a ridiculous scenario. If you have some files on your laptop that are super important, you probably shouldn't `rm -rf` them 1.5 seconds after uploading to your NAS. There is nothing ridiculous about a hard drive breaking between backups. There are situations where loosing all data after latest backup would be, if not unrecoverable, at least extremely irritating. Having redundant drives in a laptop might not be feasible, but dedicated workstations are still used in many places. > If you really are going to immediately purge some critical data from its source location 1.5 seconds after fsyncing to your drive, you should be sending it to multiple locations and waiting for confirmation containing a valid checksum before considering the write a success. Which is exactly what ZFS/btrfs in a mirrored RAID configuration does.

jrockway 11y ago

I tend to agree. RAID 1 is fine for your desktop where you don't want to waste time restoring your backup when a drive inevitably dies. When the kernel gets confused and says both drives are corrupted simultaneously, it's easy to manually repair with dd. (See which block has unrecoverable errors and copy that one over from your other drive. Repaired. Not sure why the kernel can't do this automatically. The data is gone, you can't make it worse.) For datacenter applications, having everything on one machine behind a filesystem abstraction seems like unnecessary complexity, not to mention a single point of failure and a waste of resources. I'd much rather handle things on the application level; if I want three copies of my data around, write it to three different machines with disks. When a copy seems unavailable, make an extra to keep the redundancy level the same across disk failures. If I want some redundancy encoding, just store that instead of the raw file. Now there is no complicated recovery when your disk fails. The system is always in a recovered state because disk failures are expected and uneventful. That said, there is a bit of complexity in implementing this, and even more complexity when you want all the pleasantries of a UNIX filesystem (permissions, quota, etc.). I know of only one working implementation of this technique, and it's not open source. And, of course, if you're going to use existing software rather than writing your own, you're going to need a UNIX filesystem for it to use, not some clever chunk storage manager you wrote. So btrfs and zfs persist.

skorgu 11y ago

There are lots of areas in which the data is reproducible but local continuity (and checksumming!) are still extremely valuable. Think things like large iso files, dvd rips or even intermediate stages of data analysis. Since I can reproduce them having a backup might not make sense but I still don't really /want/ to have to go to all that effort. Especially things like scratch space for analysis you really don't want bit flips since you've maybe silently invalidated later stages.

oconnore 11y ago
Did you read the article? Did you read my post? Who are you replying to? Checksumming is great. Improving uptime (in your case, by not having to recreate the data) is great. Why do you assume I don't agree with you?
- skorgu 11y ago
  "They seem to care about their data somewhat, which of course means they have off-site backups." I don't think a reply of "there are classes of data that one can care about and still not have backups of" is unreasonable. "...I can surmise is that they're confused" Again, I think showing a few cases in which users can have perfectly non-confused reasons for using these filesystems is pretty straightforward as conversational gambits go.
  - oconnore 11y ago
    The article wasn't about using RAID, it was complaining of a lack of RAID60 and triple parity in btrfs, and a lack of volume expansion features in zfs.
    - skorgu 11y ago
      Well, yes but I don't think either of those things are really addressed by "care about data IFF backups exist". I guess I either don't get what point you're making or we're just getting distracted by phrasing.