4 ms·

I have a need to store a few hundred TB of data, and tape is extremely expensive for this, since it has a $5k+ upfront cost. So I just buy external $200 USB ha

by 323 5y ago

I have a need to store a few hundred TB of data, and tape is extremely expensive for this, since it has a $5k+ upfront cost. So I just buy external $200 USB hard drives. There is a market for something lower latency, higher capacity, but pay as you go (no upfront cost). Frankly I wouldn't mind some sort of giant disk drive, vinyl record size, if that makes sense and is lower cost/TB.

h2odragon 5y ago
Yes. There was a time when the tape drive wasn't so expensive; the densities weren't there either but that market climbed the wrong side of the curve in a couple of ways. The modern tape drives are miracles of precision and the tapes are these cool little cartridges but why? The meat and machines handling tape carts liked the of 3/4in format just fine. With modern electronics for the write heads and slightly less ruinously expensive loading and running hardware we could still get decent data density, even if we have to hunt for the tracks each tape load.
jmwilson 5y ago
It didn't used to be this way. Tape was once affordable for individuals; ca. 2000 Travan was marketed to consumers and DDS wasn't too much of a stretch for prosumers. But they're all gone now and only LTO is still standing. Given the small, cornered market segment that tape now serves, there is no incentive to bring the price down. I'd like the option of being able to manage my own backups locally instead of using cloud storage, but the economics of Glacier remain too good until the storage size is several times what I could foresee needing.
- selfhoster11 5y ago
  My tape setup cost less than a single external hard drive. Older tape techs come down in price in time.
wazoox 5y ago
My company has built something for that. Basically you have a nextcloud instance that you fill up with your data, then you ask for an LTO tape to be written with it. Repeat ad infinitum. The tapes are yours; when you buy a tape it comes with 3 free operations (read, write, checksum control). If you want your tapes back, we'll mail them to you, else we keep them in storage for you. As the tapes are written using LTFS, you can easily read them back anywhere with the proper drive. You only pay a fee for the cloud storage; it also gives you access to your archive's database (what file is on which tape, etc).
- xmodem 5y ago
  That sounds fascinating. Where can I learn more? Other than writing the same data out to multiple tapes, do you have any capacity for redundancy, should a tape go bad?
  - wazoox 5y ago
    Yes you can ask to have several copies, as you simply pay for the tapes. The archival solution can store data also on disks or any S3-compatible storage. We also implemented the ability to turn off entirely (cut the power) disk chassis when unused, to save power (using 60 or 102 drives chassis). So if you have enough data (a few hundred terabytes to petabytes) it can make sense too.
- ksec 5y ago
  Is it https://www.intellique.com https://www.intellique.com?
  - wazoox 5y ago
    You can find the archival solution at intellique.org and open a free account on demo.intellique.org . The whole solution is OSS, for obvious durability reasons.

by 323 5y ago

h2odragon 5y ago

Yes. There was a time when the tape drive wasn't so expensive; the densities weren't there either but that market climbed the wrong side of the curve in a couple of ways. The modern tape drives are miracles of precision and the tapes are these cool little cartridges but why? The meat and machines handling tape carts liked the of 3/4in format just fine. With modern electronics for the write heads and slightly less ruinously expensive loading and running hardware we could still get decent data density, even if we have to hunt for the tracks each tape load.

jmwilson 5y ago

It didn't used to be this way. Tape was once affordable for individuals; ca. 2000 Travan was marketed to consumers and DDS wasn't too much of a stretch for prosumers. But they're all gone now and only LTO is still standing. Given the small, cornered market segment that tape now serves, there is no incentive to bring the price down. I'd like the option of being able to manage my own backups locally instead of using cloud storage, but the economics of Glacier remain too good until the storage size is several times what I could foresee needing.

selfhoster11 5y ago
My tape setup cost less than a single external hard drive. Older tape techs come down in price in time.

wazoox 5y ago

My company has built something for that. Basically you have a nextcloud instance that you fill up with your data, then you ask for an LTO tape to be written with it. Repeat ad infinitum. The tapes are yours; when you buy a tape it comes with 3 free operations (read, write, checksum control). If you want your tapes back, we'll mail them to you, else we keep them in storage for you. As the tapes are written using LTFS, you can easily read them back anywhere with the proper drive. You only pay a fee for the cloud storage; it also gives you access to your archive's database (what file is on which tape, etc).

xmodem 5y ago
That sounds fascinating. Where can I learn more? Other than writing the same data out to multiple tapes, do you have any capacity for redundancy, should a tape go bad?
- wazoox 5y ago
  Yes you can ask to have several copies, as you simply pay for the tapes. The archival solution can store data also on disks or any S3-compatible storage. We also implemented the ability to turn off entirely (cut the power) disk chassis when unused, to save power (using 60 or 102 drives chassis). So if you have enough data (a few hundred terabytes to petabytes) it can make sense too.
ksec 5y ago
Is it https://www.intellique.com https://www.intellique.com?
- wazoox 5y ago
  You can find the archival solution at intellique.org and open a free account on demo.intellique.org . The whole solution is OSS, for obvious durability reasons.