4 ms·

For me it's crazy how tape vendors always advertise "compressed capacity" which is 2.5 times the actual capacity. Imagine a laptop containing a 2TB SSD getting

For me it's crazy how tape vendors always advertise "compressed capacity" which is 2.5 times the actual capacity. Imagine a laptop containing a 2TB SSD getting advertised as containing a "5TB SSD"...

mschuster91 3y ago
That's because of different expectations. For a tape drive, you don't care about latency, power efficiency, random access or write amplification, all that matters is getting as much data as possible on a tape for the lowest amount of investment into tapes. So the tapes are advertised with the amount of net data that you can fit onto it, assuming a standard distribution of compressible vs non-compressible data. For a laptop SSD, all of these matter. You can't do much compression because compression consumes a lot of power and latency upon access, some compression schemes make random access harder (i.e. compression schemes without an index where you have to scan through intermediate checkpoints or, in the worst case, sequentially through the entire media), and it may lead to write amplification as well if you need to add a piece of data in the middle of a file (basically the issue with shingle HDDs). As a result, no compression possible, and so SSDs are advertised with the raw capacity (or, in fact, they are underadvertised because SSDs need spare block capacity to account for wear).
- HPsquared 3y ago
  Compressibility depends on the data though, one person might be storing log files while another stores compressed video.
  - dabiged 3y ago
    I had a lto4 (max capacity 800gb) last year that took over 5tb of data. The data was ASCII log files from a server that had been running for years. Lots of repetition (time stamps, everything ok messages, you have mail messages) and the drive compression ate it for breakfast.
- f1refly 3y ago
  But it completely disregards that most files you'd want to backup are already compressed! Even if I where to turn on aggressive compression for my ssd despite all the inconveniences, it wouldn't change much. I'll have to agree with the op, storage capacity marketing for tape is weird if not outright a scam.
  - jl6 3y ago
    A lot of tape usage is in mainframe and traditional big iron systems where data is bulk unloaded from databases in a fairly raw format, and it’s convenient to let the tape infrastructure handle both compression and encryption.
- loeg 3y ago
  > assuming a standard distribution of compressible vs non-compressible data. There's no such thing as a standard distribution of compressible data. Customer data varies wildly.
londons_explore 3y ago
A lot of business data will be encrypted by policy before it gets anywhere near the (perhaps offsite) backup system. That means any compression the tape drive offers will be useless.
- chongli 3y ago
  Why not compress the data before encrypting it? Seems like a really easy to implement solution that would speed up backup and recovery and cost almost nothing to implement.
  - teraflop 3y ago
    Yes of course, but that still means the compression offered by the tape drive is ineffective, even though that compression is assumed as part of the advertised tape capacity.
    - Karunamon 3y ago
      Even last generation tape drives have hardware encryption built in. And it doesn't make sense to me that they would encrypt, then compress. That would completely invalidate the compression for absolutely no benefit.
      - count 3y ago
        The benefit being they can sell more tapes...
      - londons_explore 3y ago
        Nobody trusts encryption built into tape drives for the same reason nobody trusts encryption built into hard drives. Even things like Windows Bitlocker and LUKS/dmcrypt on Linux totally ignore the drives ability to do any encryption, and do all encryption using the CPU before the drive sees the data. Bitlocker/LUKS could easily just calculate the encryption keys and send them to the drive and trust it to encrypt/decrypt data for them... but they don't.
Accujack 3y ago
Tape drives like LTOs use some fairly good hardware compression... believe it or not, it's significantly better than the average zip program. Depending on the data sent to the tape, it can achieve 50-60% reduction in size.

For me it's crazy how tape vendors always advertise "compressed capacity" which is 2.5 times the actual capacity. Imagine a laptop containing a 2TB SSD getting advertised as containing a "5TB SSD"...

mschuster91 3y ago

That's because of different expectations. For a tape drive, you don't care about latency, power efficiency, random access or write amplification, all that matters is getting as much data as possible on a tape for the lowest amount of investment into tapes. So the tapes are advertised with the amount of net data that you can fit onto it, assuming a standard distribution of compressible vs non-compressible data. For a laptop SSD, all of these matter. You can't do much compression because compression consumes a lot of power and latency upon access, some compression schemes make random access harder (i.e. compression schemes without an index where you have to scan through intermediate checkpoints or, in the worst case, sequentially through the entire media), and it may lead to write amplification as well if you need to add a piece of data in the middle of a file (basically the issue with shingle HDDs). As a result, no compression possible, and so SSDs are advertised with the raw capacity (or, in fact, they are underadvertised because SSDs need spare block capacity to account for wear).

HPsquared 3y ago
Compressibility depends on the data though, one person might be storing log files while another stores compressed video.
- dabiged 3y ago
  I had a lto4 (max capacity 800gb) last year that took over 5tb of data. The data was ASCII log files from a server that had been running for years. Lots of repetition (time stamps, everything ok messages, you have mail messages) and the drive compression ate it for breakfast.
f1refly 3y ago
But it completely disregards that most files you'd want to backup are already compressed! Even if I where to turn on aggressive compression for my ssd despite all the inconveniences, it wouldn't change much. I'll have to agree with the op, storage capacity marketing for tape is weird if not outright a scam.
- jl6 3y ago
  A lot of tape usage is in mainframe and traditional big iron systems where data is bulk unloaded from databases in a fairly raw format, and it’s convenient to let the tape infrastructure handle both compression and encryption.
loeg 3y ago
> assuming a standard distribution of compressible vs non-compressible data. There's no such thing as a standard distribution of compressible data. Customer data varies wildly.

londons_explore 3y ago

A lot of business data will be encrypted by policy before it gets anywhere near the (perhaps offsite) backup system. That means any compression the tape drive offers will be useless.

chongli 3y ago
Why not compress the data before encrypting it? Seems like a really easy to implement solution that would speed up backup and recovery and cost almost nothing to implement.
- teraflop 3y ago
  Yes of course, but that still means the compression offered by the tape drive is ineffective, even though that compression is assumed as part of the advertised tape capacity.
  - Karunamon 3y ago
    Even last generation tape drives have hardware encryption built in. And it doesn't make sense to me that they would encrypt, then compress. That would completely invalidate the compression for absolutely no benefit.
    - count 3y ago
      The benefit being they can sell more tapes...
    - londons_explore 3y ago
      Nobody trusts encryption built into tape drives for the same reason nobody trusts encryption built into hard drives. Even things like Windows Bitlocker and LUKS/dmcrypt on Linux totally ignore the drives ability to do any encryption, and do all encryption using the CPU before the drive sees the data. Bitlocker/LUKS could easily just calculate the encryption keys and send them to the drive and trust it to encrypt/decrypt data for them... but they don't.

Accujack 3y ago

Tape drives like LTOs use some fairly good hardware compression... believe it or not, it's significantly better than the average zip program. Depending on the data sent to the tape, it can achieve 50-60% reduction in size.