5 ms·

Tape also has some inherent disadvantages. Perhaps surprisingly, the lack of near-instantaneous seek to the data you want isn't one of them. The larger issue at

Tape also has some inherent disadvantages. Perhaps surprisingly, the lack of near-instantaneous seek to the data you want isn't one of them. The larger issue at scale is that it's not disk. It's a separate kind of device, with its own power and (often very stringent) environmental requirements, and its own software. Also big silos aren't cheap, both because of inherent cost and for lack of significant competition. Hyperscalers hate that type of heterogeneity, so they'll only go down that road if they feel they have no other choice. Yes, media are cheap, but the extra operational complexity can obliterate that advantage. A lot of this doesn't apply as you go smaller, but also the drive cost starts to dominate (can't be amortized over many cheap tapes). There's a pretty narrow band where tape is likely to have a real advantage over super-high-capacity disks.

m0lecules 5y ago
Why can't tape be self-contained like HDDs? Aside from the fact that most people using tape for archival storage don't want to pay extra for the read/write heads, SATA interface, etc., there is no reason why you couldn't package all these things into a self-contained tape unit with a small flash disk acting as a small cache and directory listing. You could definitely package such a thing for consumers, for example, but most workloads there aren't a great fit for the medium. Basically the only thing that makes sense is using it for archival and backups.
- dragontamer 5y ago
  > Why can't tape be self-contained like HDDs? Because that gets rid of the main advantage of tape. Which is that the tape-media has no read/write head and is therefore much much much cheaper to mass produce. --------- In practice, people buy tape-libraries entirely. Like a 3U unit with 50-tape slots + a few drives to read/write to those tapes, and then hook them up to the network. https://www.quantum.com/en/products/tape-storage/ https://www.quantum.com/en/products/tape-storage/ From this perspective, you buy as many tapes as you want storage (aiming for 500TB? Buy like 40 LTO7 tapes for your library. Aiming for 1000TB? Buy 80 LTO7 tapes for your library, assuming compression of course). From there, you just read/write to the library, and have the underlying software handle the details, like any other NAS.
  - riobard 5y ago
    > assuming compression of course I don't know why this seems the standard practice in the industry, but it really annoyed me when I realized a “15TB” LTO-7 tape has actually only 6TB real, “native” storage coz it assumes some average compression ratio. Why is this acceptable? What if I use the tape to store incompressible data like video and images? Feels like intentional cheating.
    - jlarocco 5y ago
      Meh. It's only cheating if the manufacturer keeps it a secret, and they don't. When a company is spending >$20k on a tape system, the people in charge of buying it will talk to the sales people, tell them the use case, and get a more accurate estimate.
      - loves_mangoes 5y ago
        If the buyers are engineers that understand the concept of storage space measured in bytes, this 'estimate' does not help them. If they don't, it only serves to mislead. The fact that buyers may talk to salespeople is really not an excuse for the deceptive behavior. "well, in our restaurant medium rare means well done and rare means medium rare, but that's okay our customers are well-paid professionals, they'll talk to waiters to get a more accurate picture"
        gowld 5y ago
        Tangent: That's how most restaurants operate, because most people don't know what the words mean and get upset when they ask for "rare" and get what they asked for. Clothing manufactures also lie about the waist measurements of pants. (Go measure yours and see.)
  - GekkePrutser 5y ago
    Funnily enough the first hard drives were like this too. They were a round pizza-sized (but a few inches thick) replaceable cartridge and could be inserted into a 'reader' which contained the heads. But I guess work the data density required these days and the extreme closeness of the head to the platters this won't work anymore as dust would get in. But this is how the first hard drives worked, we had some with our pdp-11 at the computer museum.
pradn 5y ago
If you're actually hyperscale, say 100 EB of tape, then you'll have dedicated teams just for tape archives. The heterogeneity doesn't seem like much of a problem when you have 200 engineers x 300k USD per engineer to spend on managing the system. If anything, we're seeing hyperscalers becoming more heterogeneous with CPUs. Google has training and inference TPUs, custom silicon for encoding media, and many more custom CPUs. It makes sense for storage if the benefit is there.
- notacoward 5y ago
  Those are completely different levels of heterogeneity. A GPU/TPU might draw a lot of power, but does it have to be separate power? Do you need to put that stuff on a separate slab to isolate it from vibration? Does it need its own air filtration system? Do you need a new runbook, because your usual redundancy approaches can't be used for million-dollar systems? Do your DC techs need new training, because the usual "yank and replace" isn't applicable either? I've watched colleagues in a team adjacent to my own work through these issues. Have you? Slapping a commodity card into a box and loading some commodity software on it seems like a cakewalk by comparison. Obviously people thought it was worth it anyway, but I think you're seriously misunderstanding where the difficulties lie and what kind of resources are needed to overcome them. 200+ engineers would be overkill (unless FB engineers are ~4x more productive than Google engineers) but you'll need other kinds of specialists as well and the installation/operational costs will still be high. It's unlikely to be worth it for an organization much smaller or different.
  - pradn 5y ago
    Yes, these factors would probably limit massive tape archives to only the biggest companies.

m0lecules 5y ago

Why can't tape be self-contained like HDDs? Aside from the fact that most people using tape for archival storage don't want to pay extra for the read/write heads, SATA interface, etc., there is no reason why you couldn't package all these things into a self-contained tape unit with a small flash disk acting as a small cache and directory listing. You could definitely package such a thing for consumers, for example, but most workloads there aren't a great fit for the medium. Basically the only thing that makes sense is using it for archival and backups.

dragontamer 5y ago
> Why can't tape be self-contained like HDDs? Because that gets rid of the main advantage of tape. Which is that the tape-media has no read/write head and is therefore much much much cheaper to mass produce. --------- In practice, people buy tape-libraries entirely. Like a 3U unit with 50-tape slots + a few drives to read/write to those tapes, and then hook them up to the network. https://www.quantum.com/en/products/tape-storage/ https://www.quantum.com/en/products/tape-storage/ From this perspective, you buy as many tapes as you want storage (aiming for 500TB? Buy like 40 LTO7 tapes for your library. Aiming for 1000TB? Buy 80 LTO7 tapes for your library, assuming compression of course). From there, you just read/write to the library, and have the underlying software handle the details, like any other NAS.
- riobard 5y ago
  > assuming compression of course I don't know why this seems the standard practice in the industry, but it really annoyed me when I realized a “15TB” LTO-7 tape has actually only 6TB real, “native” storage coz it assumes some average compression ratio. Why is this acceptable? What if I use the tape to store incompressible data like video and images? Feels like intentional cheating.
  - jlarocco 5y ago
    Meh. It's only cheating if the manufacturer keeps it a secret, and they don't. When a company is spending >$20k on a tape system, the people in charge of buying it will talk to the sales people, tell them the use case, and get a more accurate estimate.
    - loves_mangoes 5y ago
      If the buyers are engineers that understand the concept of storage space measured in bytes, this 'estimate' does not help them. If they don't, it only serves to mislead. The fact that buyers may talk to salespeople is really not an excuse for the deceptive behavior. "well, in our restaurant medium rare means well done and rare means medium rare, but that's okay our customers are well-paid professionals, they'll talk to waiters to get a more accurate picture"
      - gowld 5y ago
        Tangent: That's how most restaurants operate, because most people don't know what the words mean and get upset when they ask for "rare" and get what they asked for. Clothing manufactures also lie about the waist measurements of pants. (Go measure yours and see.)
- GekkePrutser 5y ago
  Funnily enough the first hard drives were like this too. They were a round pizza-sized (but a few inches thick) replaceable cartridge and could be inserted into a 'reader' which contained the heads. But I guess work the data density required these days and the extreme closeness of the head to the platters this won't work anymore as dust would get in. But this is how the first hard drives worked, we had some with our pdp-11 at the computer museum.

pradn 5y ago

If you're actually hyperscale, say 100 EB of tape, then you'll have dedicated teams just for tape archives. The heterogeneity doesn't seem like much of a problem when you have 200 engineers x 300k USD per engineer to spend on managing the system. If anything, we're seeing hyperscalers becoming more heterogeneous with CPUs. Google has training and inference TPUs, custom silicon for encoding media, and many more custom CPUs. It makes sense for storage if the benefit is there.

notacoward 5y ago
Those are completely different levels of heterogeneity. A GPU/TPU might draw a lot of power, but does it have to be separate power? Do you need to put that stuff on a separate slab to isolate it from vibration? Does it need its own air filtration system? Do you need a new runbook, because your usual redundancy approaches can't be used for million-dollar systems? Do your DC techs need new training, because the usual "yank and replace" isn't applicable either? I've watched colleagues in a team adjacent to my own work through these issues. Have you? Slapping a commodity card into a box and loading some commodity software on it seems like a cakewalk by comparison. Obviously people thought it was worth it anyway, but I think you're seriously misunderstanding where the difficulties lie and what kind of resources are needed to overcome them. 200+ engineers would be overkill (unless FB engineers are ~4x more productive than Google engineers) but you'll need other kinds of specialists as well and the installation/operational costs will still be high. It's unlikely to be worth it for an organization much smaller or different.
- pradn 5y ago
  Yes, these factors would probably limit massive tape archives to only the biggest companies.