4 ms·

This capability is why I've always regarded ZFS as great for people who buy drives by the dozen, but a fairly poor choice for consumers and hobbyists. The abili

This capability is why I've always regarded ZFS as great for people who buy drives by the dozen, but a fairly poor choice for consumers and hobbyists. The ability to rebalance a btrfs array and switch to and from any of the RAID modes makes it vastly easier to cobble together useful redundancy from miscellaneous drives, or to recover from a drive failure without having to immediately go out and buy a suitably large replacement drive. If btrfs RAID5/6 hadn't taken so long to get to the current barely-usable-with-caveats state, I doubt ZFS could have caught on among home Linux users.

hinkley 6y ago
We need a consumer-grade NAS where you can push a button, pop out the crappiest drive, shove in a new drive, and after an hour it goes 'ding' and you are ready to go to the family reunion and take a crapload of pictures. Those are, of course, never going to be an all-drives-at-once affair like you illustrate. I wish Synology or Drobo had invested in solving this problem for everyone instead of doing their own proprietary solutions. Drobo has a whole filesystem, Synology has logical layer on top of RAID and/or btrfs to simulate this for some configurations of disk. In both cases, slapping a single higher capacity drive into the array gets you no increase in disk capacity. So every nth upgrade you are looking at 2 sequential rebuilds.
- Someone 6y ago
  “after an hour it goes 'ding'” Most modern hard disks are too large/too slow for that. A write speed of 200MB/s translates to 12GB/minute, or 720GB/hour. So, a 2TB hard disk (‘small’ for today) already would take 2½ hours to fill, and that assumes you hit 200MB/s continuously. For a 12TB disk, “after a _day_ it goes 'ding'” would be closer to the truth. I also think that, for most consumers, storage in the cloud is a better option than a “consumer-grade NAS”.
  - hinkley 6y ago
    With my synology it was faster to back up the array, break the volume, recreate it and copy the data back than it was to swap the 2nd of 3 disks I was trying to upgrade. The first ding was 6 hours, the second was on track for 48 hours, and I had another one to go. It shouldn't be like this. I'm not convinced, in the specific case of a consumer system, that every track on every disk needs to be touched during a build/rebuild. It amplifies the read and write traffic at exactly the point where any traffic is a ticking time bomb waiting to take your data away (which is exactly what happened the first time I tried to upgrade the Synology - a drive started reporting bad sectors during the verification after rebuild, but before marking the array healthy). In a production server you might be more uncomfortable with a drive array that experiences an amortized cost spread over every read or write operation, especially one that upticks every time your total storage exceeds the previous high water mark. But even there, I think I could make a reasonable if not airtight case for that as well. Last conversation, I believe someone told me that the cost of resilvering a ZFS array is proportional to the contents and not the capacity. I don't know if that would be true in the case of the patch being discussed. The best any of us can expect is for the costs to be similar to the kind of service disruption you get when adding or removing servers from a cluster that is using consistent hashing with m-way redundancy. There are always m machines involved in every write, and you are going to have to move 1 to 2 nths of the data every time n changes, and that may be as high as m/n reads and writes. But 2/5th is a lot smaller than 3 reads, 1 write, 3 reads per block for 100% of the blocks in the volume. But I haven't seen a system that is anywhere near that minimum cost.
  - boring_twenties 6y ago
    Where I live, "most consumers" are stuck with Comcast which features a 2MByte upload speed (at very best -- currently I'm getting <1MB), and a 1TB monthly transfer limit. Cloud storage is a non-option.
    - Someone 6y ago
      I still disagree. Running with only cloud storage wouldn’t work, but a local SSD and cloud storage, why not? For “most consumers”, I would think the data they create mostly is created in their phones (photos, videos), anyways, and I don’t see them syncing that to their local NAS, instead of using the more convenient sync to the cloud.
      - boring_twenties 6y ago
        I guess I don't know how much phone videos take up, but I know that when I was using my GoPro I was able to fill up several hundred GB in a few weeks. And that was with an ancient 720p one. For photos, I suppose it would be good enough. Actually, perversely enough my phone does have unlimited data via its LTE connection. If only phones had the option to "download over cellular network only" instead of the reverse!

hinkley 6y ago

We need a consumer-grade NAS where you can push a button, pop out the crappiest drive, shove in a new drive, and after an hour it goes 'ding' and you are ready to go to the family reunion and take a crapload of pictures. Those are, of course, never going to be an all-drives-at-once affair like you illustrate. I wish Synology or Drobo had invested in solving this problem for everyone instead of doing their own proprietary solutions. Drobo has a whole filesystem, Synology has logical layer on top of RAID and/or btrfs to simulate this for some configurations of disk. In both cases, slapping a single higher capacity drive into the array gets you no increase in disk capacity. So every nth upgrade you are looking at 2 sequential rebuilds.

Someone 6y ago
“after an hour it goes 'ding'” Most modern hard disks are too large/too slow for that. A write speed of 200MB/s translates to 12GB/minute, or 720GB/hour. So, a 2TB hard disk (‘small’ for today) already would take 2½ hours to fill, and that assumes you hit 200MB/s continuously. For a 12TB disk, “after a _day_ it goes 'ding'” would be closer to the truth. I also think that, for most consumers, storage in the cloud is a better option than a “consumer-grade NAS”.
- hinkley 6y ago
  With my synology it was faster to back up the array, break the volume, recreate it and copy the data back than it was to swap the 2nd of 3 disks I was trying to upgrade. The first ding was 6 hours, the second was on track for 48 hours, and I had another one to go. It shouldn't be like this. I'm not convinced, in the specific case of a consumer system, that every track on every disk needs to be touched during a build/rebuild. It amplifies the read and write traffic at exactly the point where any traffic is a ticking time bomb waiting to take your data away (which is exactly what happened the first time I tried to upgrade the Synology - a drive started reporting bad sectors during the verification after rebuild, but before marking the array healthy). In a production server you might be more uncomfortable with a drive array that experiences an amortized cost spread over every read or write operation, especially one that upticks every time your total storage exceeds the previous high water mark. But even there, I think I could make a reasonable if not airtight case for that as well. Last conversation, I believe someone told me that the cost of resilvering a ZFS array is proportional to the contents and not the capacity. I don't know if that would be true in the case of the patch being discussed. The best any of us can expect is for the costs to be similar to the kind of service disruption you get when adding or removing servers from a cluster that is using consistent hashing with m-way redundancy. There are always m machines involved in every write, and you are going to have to move 1 to 2 nths of the data every time n changes, and that may be as high as m/n reads and writes. But 2/5th is a lot smaller than 3 reads, 1 write, 3 reads per block for 100% of the blocks in the volume. But I haven't seen a system that is anywhere near that minimum cost.
- boring_twenties 6y ago
  Where I live, "most consumers" are stuck with Comcast which features a 2MByte upload speed (at very best -- currently I'm getting <1MB), and a 1TB monthly transfer limit. Cloud storage is a non-option.
  - Someone 6y ago
    I still disagree. Running with only cloud storage wouldn’t work, but a local SSD and cloud storage, why not? For “most consumers”, I would think the data they create mostly is created in their phones (photos, videos), anyways, and I don’t see them syncing that to their local NAS, instead of using the more convenient sync to the cloud.
    - boring_twenties 6y ago
      I guess I don't know how much phone videos take up, but I know that when I was using my GoPro I was able to fill up several hundred GB in a few weeks. And that was with an ancient 720p one. For photos, I suppose it would be good enough. Actually, perversely enough my phone does have unlimited data via its LTE connection. If only phones had the option to "download over cellular network only" instead of the reverse!