4 ms·

I would highly recommend jacking up the --blocking-factor on tar (or your custom software) as high as your drive will accept. AFAIK the default on tar is 512b w

I would highly recommend jacking up the --blocking-factor on tar (or your custom software) as high as your drive will accept. AFAIK the default on tar is 512b which is utterly inadequate, and even sequential data on a fast SSD will shoe-shine when written directly to tape. AFAIK most large businesses wind up writing to disk and then queuing the resulting serialized virtual tapes for actual writing at a later time. This basically turns the virtual tape library into a massive write buffer. It's absolutely insane that we have to do this - but a lot of archival utilities aren't built for async/parallel IO and thus aren't taking advantage of modern storage devices.

sly010 6y ago
The tool I ended up writing does essentially something very similar: - It uses a block size that matches the disk - It uses 50% of my RAM as a ring buffer - On start (and on buffer underrun) it waits for the buffer to fill before it starts writing to tape. - It adjust the tape write speed as an exponential function of the buffer fill level. I am proud of this last one. It is very simple and automatically converges to the best average speed to keep both the tape speed and the buffer level relatively stable. I have been considering making the reading multi threaded to make use of more disks, but that would be a bit of an overkill for my personal 2 disk NAS. edit:formatting
- js2 6y ago
  - How many TB are you backing up? - How frequently do you perform backups? - How often do you do fulls vs incremental? - What drive/autoloader are you using, and what was the total investment in drive + tapes? I keep considering a used LTO but I've only got 20 TB and it doesn't seem worth the trouble. I should just buy another 20 TB of drives and keep them offline when not backing up.
  - foobiekr 6y ago
    I back up around 20TB and the easiest solution I found was to just buy 4x12TB drives and mirror to cloud (yielding 2x offline copies). For stuff that's really really important, around half of that, and which if restore is needed fast access is required, I also back it up to SSD. It's not the greatest solution, but when you're hovering at some small multiple of common single drives, it's certainly the easy answer.
  - sly010 6y ago
    I only have ~1.5TB stuff that is really important which fits on a single LTO5 tape. (I said LTO6 before, I was mistaken) I archive every ~3 months, generally before I make bigger changes. I always do a full tar archive, it typically takes the whole day, definitely an event my family is aware of. I use a desktop HP Ultrium LTO5 drive. I fit on a single tape, but even if I didn't, changing tape twice a day would not be an big ask. Months of researching, ~$900 total on 2 used drives (first one was broken) + enclosure + SAS controller + 5 tapes. You need the enclosure with forced air cooling, you can't put the bare drive in a PC tower, it will overheat. I cannot imagine archiving 20TB with my cheapo setup, but I also cannot imagine spending ~3K for a newer drive, this is a sweet spot for me. After buying a broken drive first, I am now always nervous that this drive will quit on me. It is definitely a passion project at this point.
    - js2 6y ago
      Thanks for the info. For 1.5 TB I'd just use an extra drive. My 20 TB is 11 x 3 TB used SAS drives in a raidz3 config. You can pickup a 3 TB drive off eBay for $18-$24. I should've probably just bought four 10 TB WD red drives, but building this system has been, uh, a passion project I guess. :-) It's still cheaper than if I'd bought new red drives, but I haven't factored in the electricity cost of running 11 SAS drives vs four red drives... the breakeven is probably less than a year. :-(
- axiolite 6y ago
  Sounds like you reinvented mbuffer. http://www.maier-komor.de/mbuffer.html http://www.maier-komor.de/mbuffer.html Why not publish yours as well? For other file systems (not ZFS), spd_readdir.so can offer a good performance boost. https://kernel.googlesource.com/pub/scm/linux/kernel/git/djwong/e2fsprogs/+/748924621fdd0e70a191091915fc9724d42e4796/contrib/spd_readdir.c https://kernel.googlesource.com/pub/scm/linux/kernel/git/djw... It's probably possible to do the same on ZFS, but CoW certainly makes realizing this concept more difficult.
swiley 6y ago
Tar not working for modern tapes by default is pretty ironic.
- vram22 6y ago
  Not necessarily. Tar was invented way back, in early Unix days. I know it would have been updated sometimes, in GNU tar and other Unix versions, but they may not have anticipated the massive growth in tape capacities and speeds. Using dd with a high block size to write "virtual tapes" (as described elsewhere in this thread) from disk to tape might be one solution (not tried, just had the idea now).

sly010 6y ago

The tool I ended up writing does essentially something very similar: - It uses a block size that matches the disk - It uses 50% of my RAM as a ring buffer - On start (and on buffer underrun) it waits for the buffer to fill before it starts writing to tape. - It adjust the tape write speed as an exponential function of the buffer fill level. I am proud of this last one. It is very simple and automatically converges to the best average speed to keep both the tape speed and the buffer level relatively stable. I have been considering making the reading multi threaded to make use of more disks, but that would be a bit of an overkill for my personal 2 disk NAS. edit:formatting

js2 6y ago
- How many TB are you backing up? - How frequently do you perform backups? - How often do you do fulls vs incremental? - What drive/autoloader are you using, and what was the total investment in drive + tapes? I keep considering a used LTO but I've only got 20 TB and it doesn't seem worth the trouble. I should just buy another 20 TB of drives and keep them offline when not backing up.
- foobiekr 6y ago
  I back up around 20TB and the easiest solution I found was to just buy 4x12TB drives and mirror to cloud (yielding 2x offline copies). For stuff that's really really important, around half of that, and which if restore is needed fast access is required, I also back it up to SSD. It's not the greatest solution, but when you're hovering at some small multiple of common single drives, it's certainly the easy answer.
- sly010 6y ago
  I only have ~1.5TB stuff that is really important which fits on a single LTO5 tape. (I said LTO6 before, I was mistaken) I archive every ~3 months, generally before I make bigger changes. I always do a full tar archive, it typically takes the whole day, definitely an event my family is aware of. I use a desktop HP Ultrium LTO5 drive. I fit on a single tape, but even if I didn't, changing tape twice a day would not be an big ask. Months of researching, ~$900 total on 2 used drives (first one was broken) + enclosure + SAS controller + 5 tapes. You need the enclosure with forced air cooling, you can't put the bare drive in a PC tower, it will overheat. I cannot imagine archiving 20TB with my cheapo setup, but I also cannot imagine spending ~3K for a newer drive, this is a sweet spot for me. After buying a broken drive first, I am now always nervous that this drive will quit on me. It is definitely a passion project at this point.
  - js2 6y ago
    Thanks for the info. For 1.5 TB I'd just use an extra drive. My 20 TB is 11 x 3 TB used SAS drives in a raidz3 config. You can pickup a 3 TB drive off eBay for $18-$24. I should've probably just bought four 10 TB WD red drives, but building this system has been, uh, a passion project I guess. :-) It's still cheaper than if I'd bought new red drives, but I haven't factored in the electricity cost of running 11 SAS drives vs four red drives... the breakeven is probably less than a year. :-(
axiolite 6y ago
Sounds like you reinvented mbuffer. http://www.maier-komor.de/mbuffer.html http://www.maier-komor.de/mbuffer.html Why not publish yours as well? For other file systems (not ZFS), spd_readdir.so can offer a good performance boost. https://kernel.googlesource.com/pub/scm/linux/kernel/git/djwong/e2fsprogs/+/748924621fdd0e70a191091915fc9724d42e4796/contrib/spd_readdir.c https://kernel.googlesource.com/pub/scm/linux/kernel/git/djw... It's probably possible to do the same on ZFS, but CoW certainly makes realizing this concept more difficult.

swiley 6y ago

Tar not working for modern tapes by default is pretty ironic.

vram22 6y ago
Not necessarily. Tar was invented way back, in early Unix days. I know it would have been updated sometimes, in GNU tar and other Unix versions, but they may not have anticipated the massive growth in tape capacities and speeds. Using dd with a high block size to write "virtual tapes" (as described elsewhere in this thread) from disk to tape might be one solution (not tried, just had the idea now).