4 ms·

Tape robot time is NOT measured in minutes but hours. Also you wouldn't pull individual files like this out of tape. Tape is for long term sunk storage, not co

Tape robot time is NOT measured in minutes but hours. Also you wouldn't pull individual files like this out of tape. Tape is for long term sunk storage, not cold infrequent access like a youtube video. I know aws glacier has an "expedited retrieval time" of 1-5 minutes, but that is not how typical tape setups work. Frankly I would be very interested in what actually hides behind that product.

joking 2y ago
You can use big disk, but not be able to access all the files on the disk with the same frequency, so you have 20% of the disk dedicated to hot storage, and 80% of it to cold storage. Cold storage access is queued , so the 1-5 minutes can come from there.
- olivermuty 2y ago
  Yeah I looked closer and I think they are basically just packaging up some kind of offering on top of spinning rust for the two first glacier tiers.
swiftcoder 2y ago
Glacier is not actually tape, the fancy tape-robot videos nonwithstanding. Most of it is just regular old S3 running on outdated storage hardware
- remram 2y ago
  Interesting. Why does it take hours to retrieve then? Network/disk bandwidth really is that bad? Also do you have a source for this?
  - magarnicle 2y ago
    To discourage access, maybe? That pushes you to use more expensive options.
    - remram 2y ago
      This doesn't hold, not having the cheap option encourages you to use the expensive option even better...
  - swiftcoder 2y ago
    It's mostly just bandwidth prioritization, slotting large transfers in when there is excess bandwidth. You can tell this is the case for at least the flexible retrieval tier, because small objects can be returned in a few minutes, whereas larger requests take hours - if the files were actually on a tape drive somewhere, small requests couldn't be fulfilled dramatically faster than large ones, given that tape has shitty random-access performance. (I used to work on a downstream team at AWS)
    - remram 2y ago
      I believe you, I can see how it would make sense that AWS would create a tier to exploit the spare capacity in S3 disk bandwidth, just like they did for EC2 spare VM capacity with spot instances. Still it doesn't make intuitive sense to me how the performance AND the price can be so far. That's why I'd love a longer write-up if you know of any. It's also weird that the retrieval gives you a regular fast S3 object you can then access. Given that it's already on that hardware, is a copy even happening?
surajrmal 2y ago
Google gave up on tape a while back. Latest Google search indicates it is only used for air gapped backups. I don't OP was suggesting using tape though, especially with technologies such as hybrid SMR.
PhasmaFelis 2y ago
That's all true, but I don't think anyone mentioned tape storage.

joking 2y ago

You can use big disk, but not be able to access all the files on the disk with the same frequency, so you have 20% of the disk dedicated to hot storage, and 80% of it to cold storage. Cold storage access is queued , so the 1-5 minutes can come from there.

olivermuty 2y ago
Yeah I looked closer and I think they are basically just packaging up some kind of offering on top of spinning rust for the two first glacier tiers.

swiftcoder 2y ago

Glacier is not actually tape, the fancy tape-robot videos nonwithstanding. Most of it is just regular old S3 running on outdated storage hardware

remram 2y ago
Interesting. Why does it take hours to retrieve then? Network/disk bandwidth really is that bad? Also do you have a source for this?
- magarnicle 2y ago
  To discourage access, maybe? That pushes you to use more expensive options.
  - remram 2y ago
    This doesn't hold, not having the cheap option encourages you to use the expensive option even better...
- swiftcoder 2y ago
  It's mostly just bandwidth prioritization, slotting large transfers in when there is excess bandwidth. You can tell this is the case for at least the flexible retrieval tier, because small objects can be returned in a few minutes, whereas larger requests take hours - if the files were actually on a tape drive somewhere, small requests couldn't be fulfilled dramatically faster than large ones, given that tape has shitty random-access performance. (I used to work on a downstream team at AWS)
  - remram 2y ago
    I believe you, I can see how it would make sense that AWS would create a tier to exploit the spare capacity in S3 disk bandwidth, just like they did for EC2 spare VM capacity with spot instances. Still it doesn't make intuitive sense to me how the performance AND the price can be so far. That's why I'd love a longer write-up if you know of any. It's also weird that the retrieval gives you a regular fast S3 object you can then access. Given that it's already on that hardware, is a copy even happening?

surajrmal 2y ago

Google gave up on tape a while back. Latest Google search indicates it is only used for air gapped backups. I don't OP was suggesting using tape though, especially with technologies such as hybrid SMR.

PhasmaFelis 2y ago

That's all true, but I don't think anyone mentioned tape storage.