4 ms·

Finally, a use for that drawer full of 1 GB USB drives!

by eby 3y ago

o11c 3y ago
Beware that bad sectors probably exist on those, and modern filesystems (which falsely assume the hardware will do automagical remapping) make it hard to run `badblocks`. From the fsck man pages, only ext2, ext3, ext4, and reiserfs support it. I hope that `btrfs-convert` deals with it correctly, but you must not ever run `btrfs balance` or delete the backup (it won't waste space if it was a fresh FS). And if further bad blocks develop you'll have to redo this all from scratch.
- M95D 3y ago
  btrfs doesn't support bad blocks. It's written in the documentation. From experience I can tell you that most usb sticks fail while reading back data by returning bad data, not an error or timeout, and the defective sectors aren't always the same (internal remap to wear leveling might be the cause of that). Advice: Do md-raid0 on half of them, md-raid0 on the other half, and btrfs raid1 on the two md devices. If you want to exceed 25MB/s, use different USB controllers, not different ports on the same hub.
  - o11c 3y ago
    Btrfs doesn't directly support bad blocks. But since the conversion is supposed to use only "free space" that's presumably excluding bad blocks (which are presumably counted as allocated).
    - M95D 3y ago
      Doesn't support bad blocks at all. Bad blocks don't count as allocated unless they are newly discovered - data should be moved asap if it's still recoverable. An allocated block with nothing in it is an error.
__lm__ 3y ago
When Sun Microsystems was still existing they made a (quite funny) video putting 12 usb sticks in a ZFS pool with RAID: https://constantin.glez.de/2011/01/24/how-save-world-zfs-and-12-usb-sticks-4th-anniversary-video-re-release-edition/ https://constantin.glez.de/2011/01/24/how-save-world-zfs-and...
- somat 3y ago
  I wanted to boot my home truenas box off usb flash, but I also know how flakey usb flash can be, so I use a pair of drives in a zfs mirror. I would not recommend this setup, Any real drive would be better. I just hated wasting drive bays for the non-critical boot/system partition. In my defense despite burning through many cheap usb drives I have not lost the boot yet, however I do keep a new spare drive taped to the unit.
  - hdjdkdbdbe 3y ago
    I have the same setup for my router ... Openbsd 16-32gb if one dies I create a new one and set it up with ansible....
- cylinder714 3y ago
  From 2009, what I think was a famous video on which some kids in the UK put together a 6-gigabyte RAID with 24 256-gigabyte Samsung drives: https://youtu.be/26enkCzkJHQ https://youtu.be/26enkCzkJHQ
- phil21 3y ago
  I had grand plans to make a giant USB ZFS volume from tradeshow thumb drives in bulk. It lasted a couple evenings while I bumped into apparently every Linux USB driver/chipset issue possible trying to drive a couple hundred drives via cheapo AliExpress USB hubs. It was fun times, but even more useless than I originally thought it would be. It was amusing to get >1GBps for a minute or two across 256 crappy thumb drives though!

Finally, a use for that drawer full of 1 GB USB drives!

by eby 3y ago

Finally, a use for that drawer full of 1 GB USB drives!

o11c 3y ago

Beware that bad sectors probably exist on those, and modern filesystems (which falsely assume the hardware will do automagical remapping) make it hard to run `badblocks`. From the fsck man pages, only ext2, ext3, ext4, and reiserfs support it. I hope that `btrfs-convert` deals with it correctly, but you must not ever run `btrfs balance` or delete the backup (it won't waste space if it was a fresh FS). And if further bad blocks develop you'll have to redo this all from scratch.

M95D 3y ago
btrfs doesn't support bad blocks. It's written in the documentation. From experience I can tell you that most usb sticks fail while reading back data by returning bad data, not an error or timeout, and the defective sectors aren't always the same (internal remap to wear leveling might be the cause of that). Advice: Do md-raid0 on half of them, md-raid0 on the other half, and btrfs raid1 on the two md devices. If you want to exceed 25MB/s, use different USB controllers, not different ports on the same hub.
- o11c 3y ago
  Btrfs doesn't directly support bad blocks. But since the conversion is supposed to use only "free space" that's presumably excluding bad blocks (which are presumably counted as allocated).
  - M95D 3y ago
    Doesn't support bad blocks at all. Bad blocks don't count as allocated unless they are newly discovered - data should be moved asap if it's still recoverable. An allocated block with nothing in it is an error.

__lm__ 3y ago

When Sun Microsystems was still existing they made a (quite funny) video putting 12 usb sticks in a ZFS pool with RAID: https://constantin.glez.de/2011/01/24/how-save-world-zfs-and-12-usb-sticks-4th-anniversary-video-re-release-edition/ https://constantin.glez.de/2011/01/24/how-save-world-zfs-and...

somat 3y ago
I wanted to boot my home truenas box off usb flash, but I also know how flakey usb flash can be, so I use a pair of drives in a zfs mirror. I would not recommend this setup, Any real drive would be better. I just hated wasting drive bays for the non-critical boot/system partition. In my defense despite burning through many cheap usb drives I have not lost the boot yet, however I do keep a new spare drive taped to the unit.
- hdjdkdbdbe 3y ago
  I have the same setup for my router ... Openbsd 16-32gb if one dies I create a new one and set it up with ansible....
cylinder714 3y ago
From 2009, what I think was a famous video on which some kids in the UK put together a 6-gigabyte RAID with 24 256-gigabyte Samsung drives: https://youtu.be/26enkCzkJHQ https://youtu.be/26enkCzkJHQ
phil21 3y ago
I had grand plans to make a giant USB ZFS volume from tradeshow thumb drives in bulk. It lasted a couple evenings while I bumped into apparently every Linux USB driver/chipset issue possible trying to drive a couple hundred drives via cheapo AliExpress USB hubs. It was fun times, but even more useless than I originally thought it would be. It was amusing to get >1GBps for a minute or two across 256 crappy thumb drives though!