4 ms·

Up to 2 minutes seek time for cold data. Very niche.

by curiousObject 2y ago

throwaway48476 2y ago
Depends on price. If it was good enough I'd use it for backups.
- loeg 2y ago
  It has to cost more than tape alone with the flash media sitting in front of it.
  - Etheryte 2y ago
    That's true, but this is considerably more convenient than traditional tape. Tape as is requires specialty equipment and careful handling, if I understand the above product proposition correctly, it's just plug and play. Right now, only an enthusiast would use tape in their home setup, with this approach, anyone could use it for their backup server or w/e.
    - b112 2y ago
      Meanwhile, some guide online (probably AI crafted), mixes up swap space and cold storage space. A new swap of death.
  - ramses0 2y ago
    But, "never underestimate the bandwidth of a station wagon full of hard drives". The intriguing use case (thinking as a consumer) is ~2tb "hot", and effectively treating the remainder tape as kindof "append-only/WAL" (and: e.g. incremental backups). I have been randomly ripping my DVDs, mostly keeping the transcoded MP4s, but hesitant to delete the ISO's because they require physical handling and _time_ to re-rip. If it's ~2min to seek to an 8gb ISO, that's potentially faster than re-downloading it from somewhere offsite, and would match my hypothetical use cases really well. Keeping a local ~2tb drive backed up to a ~4tb+40tb "ssd-tape-hard-drive" is probably cheaper than most long-term cloud stuff (backblaze ~$6/tb/mo!!, glacier ~$4/tb/mo, but with lots of asterisks on retrieval time), and simpler than messing around with a NAS. Glad I don't truly have these problems!
- knowitnone 2y ago
  China will sell this to you as a service
fnordpiglet 2y ago
It’s niche enough to be a major industry. Cold storage at extreme scales for very low cost is higher value than it appears - there’s a lot of data that isn’t worth very much individually but has extremely high value if it’s ever needed - but you can’t know ahead of time which data that is. That’s all about ultra low price per gb storage that’s high durable.
- curiousObject 2y ago
  This idea adds a fast SSD cache to a slow access bulk archive. Obviously that cache can massively accelerate repeated access to the same data, and reduce tape wear and mechanical wear and energy usage. But I think the critical issue is where you want the cache to be. Because end point devices already have local cache. So this product only seems to have a big advantage on applications where you expect many unique users to access the same archive data for the first time, and within a short period of time before it expires from the tape drive’s SSD cache. This is why I think it’s a small niche. Maybe too small to be worthwhile. Edit: it also has the disadvantage that the first few devices to try to access the data would have a huge delay for a minute or more while the data is located on the tape. It’s also vulnerable to DDOS by attackers requesting random cold data, to fill the SSD.
  - magicalhippo 2y ago
    Got me thinking if anyone tried something ala torrents for utilizing client-side caches to lighten the load on the server. If the alternative is slow storage, getting it from a different client on the LAN might be win.
dartos 2y ago
Well yeah, it’s tape storage.
petra 2y ago
If you want to build something like youtube, you could store the data from a start of a video, and data defining a slower resolution stream, and more popular videos on the lower latency parts of tapes in your storage arrays, and while the video plays you could go get the rest from the higher latency places. You could also predict What is the next video the user will want. Also, the 2 minute seek time is from one end of the tape to the other. But you could also move accross tracks, vertically so that gives another route for short seek times for some of your data. So it might be possible to build something with a reasonable performance.
- ByThyGrace 2y ago
  Sounds like minute long ads are inevitable and unskippable while the system fetches the video file.
  - shiroiushi 2y ago
    They'd probably have an easier time getting people to watch ads if they had a message: "please enjoy this message from our sponsor while we wait for your video to be retrieved from the offline archives".
maccard 2y ago
My workplace archived old builds of our product. It’s very rare that we need them, but we do need them occasionally. We’d happily take 1 hour latency to be able to cheaply store more of them.
- baq 2y ago
  Almost literally designed for you: https://aws.amazon.com/s3/storage-classes/glacier/ https://aws.amazon.com/s3/storage-classes/glacier/
  - Dylan16807 2y ago
    It is, and google and microsoft have similar. But whether it's the best option is a big maybe. Glacier is great at smaller scales, or to make things simple. (Well, great excluding the egress prices.) But by the time your data won't fit on a large NAS (>200TB), a $5000 tape drive and a bunch of $5/TB tapes are cheaper than glacier.
gwbas1c 2y ago
Seems like a great way to: - Archive for the film industry. 2 minutes to retrieve an archived take is a good compromise of cost and time. - Archive for the recording industry. 2 minutes to recover the "master tapes" when you remaster for the umpteenth time is a good compromise of cost and time. - The product I work on has requirements to archive data for a year / 10 years / 20 years and then generate a report. A 2 minute delay is perfectly acceptable.
jancsika 2y ago
That's plenty quick enough for the Windows Start button. Zing!
White_Wolf 2y ago
That is not niche at all. Pretty much any gov't, corp, etc that needs long term achives could do with faster access. 2 min is light speed since it's low(er) cost/decade/gb
Ekaros 2y ago
I see lot of reasonable use cases, say something like Internet Archive or library in general. Keep the frequented stuff in more hot cache and then rest of the stuff in colder storage. Without having to actually think about the process. Similar could be some new media production company. Archive B-roll or parts of older videos and then have the stuff you use very common available much faster.
lightedman 2y ago
This could be easily improved by keeping track of what's on the tape where in part of the SSD, like a length-based filesystem. Then it is just a matter of seeking to that length of tape and finding the actual start of the data. You could take the seek time for cold data to just a couple dozen seconds.

Up to 2 minutes seek time for cold data. Very niche.

throwaway48476 2y ago

Depends on price. If it was good enough I'd use it for backups.

loeg 2y ago
It has to cost more than tape alone with the flash media sitting in front of it.
- Etheryte 2y ago
  That's true, but this is considerably more convenient than traditional tape. Tape as is requires specialty equipment and careful handling, if I understand the above product proposition correctly, it's just plug and play. Right now, only an enthusiast would use tape in their home setup, with this approach, anyone could use it for their backup server or w/e.
  - b112 2y ago
    Meanwhile, some guide online (probably AI crafted), mixes up swap space and cold storage space. A new swap of death.
- ramses0 2y ago
  But, "never underestimate the bandwidth of a station wagon full of hard drives". The intriguing use case (thinking as a consumer) is ~2tb "hot", and effectively treating the remainder tape as kindof "append-only/WAL" (and: e.g. incremental backups). I have been randomly ripping my DVDs, mostly keeping the transcoded MP4s, but hesitant to delete the ISO's because they require physical handling and _time_ to re-rip. If it's ~2min to seek to an 8gb ISO, that's potentially faster than re-downloading it from somewhere offsite, and would match my hypothetical use cases really well. Keeping a local ~2tb drive backed up to a ~4tb+40tb "ssd-tape-hard-drive" is probably cheaper than most long-term cloud stuff (backblaze ~$6/tb/mo!!, glacier ~$4/tb/mo, but with lots of asterisks on retrieval time), and simpler than messing around with a NAS. Glad I don't truly have these problems!
knowitnone 2y ago
China will sell this to you as a service

fnordpiglet 2y ago

It’s niche enough to be a major industry. Cold storage at extreme scales for very low cost is higher value than it appears - there’s a lot of data that isn’t worth very much individually but has extremely high value if it’s ever needed - but you can’t know ahead of time which data that is. That’s all about ultra low price per gb storage that’s high durable.

curiousObject 2y ago
This idea adds a fast SSD cache to a slow access bulk archive. Obviously that cache can massively accelerate repeated access to the same data, and reduce tape wear and mechanical wear and energy usage. But I think the critical issue is where you want the cache to be. Because end point devices already have local cache. So this product only seems to have a big advantage on applications where you expect many unique users to access the same archive data for the first time, and within a short period of time before it expires from the tape drive’s SSD cache. This is why I think it’s a small niche. Maybe too small to be worthwhile. Edit: it also has the disadvantage that the first few devices to try to access the data would have a huge delay for a minute or more while the data is located on the tape. It’s also vulnerable to DDOS by attackers requesting random cold data, to fill the SSD.
- magicalhippo 2y ago
  Got me thinking if anyone tried something ala torrents for utilizing client-side caches to lighten the load on the server. If the alternative is slow storage, getting it from a different client on the LAN might be win.

dartos 2y ago

Well yeah, it’s tape storage.

petra 2y ago

If you want to build something like youtube, you could store the data from a start of a video, and data defining a slower resolution stream, and more popular videos on the lower latency parts of tapes in your storage arrays, and while the video plays you could go get the rest from the higher latency places. You could also predict What is the next video the user will want. Also, the 2 minute seek time is from one end of the tape to the other. But you could also move accross tracks, vertically so that gives another route for short seek times for some of your data. So it might be possible to build something with a reasonable performance.

ByThyGrace 2y ago
Sounds like minute long ads are inevitable and unskippable while the system fetches the video file.
- shiroiushi 2y ago
  They'd probably have an easier time getting people to watch ads if they had a message: "please enjoy this message from our sponsor while we wait for your video to be retrieved from the offline archives".

maccard 2y ago

My workplace archived old builds of our product. It’s very rare that we need them, but we do need them occasionally. We’d happily take 1 hour latency to be able to cheaply store more of them.

baq 2y ago
Almost literally designed for you: https://aws.amazon.com/s3/storage-classes/glacier/ https://aws.amazon.com/s3/storage-classes/glacier/
- Dylan16807 2y ago
  It is, and google and microsoft have similar. But whether it's the best option is a big maybe. Glacier is great at smaller scales, or to make things simple. (Well, great excluding the egress prices.) But by the time your data won't fit on a large NAS (>200TB), a $5000 tape drive and a bunch of $5/TB tapes are cheaper than glacier.

gwbas1c 2y ago

Seems like a great way to: - Archive for the film industry. 2 minutes to retrieve an archived take is a good compromise of cost and time. - Archive for the recording industry. 2 minutes to recover the "master tapes" when you remaster for the umpteenth time is a good compromise of cost and time. - The product I work on has requirements to archive data for a year / 10 years / 20 years and then generate a report. A 2 minute delay is perfectly acceptable.

jancsika 2y ago

That's plenty quick enough for the Windows Start button. Zing!

White_Wolf 2y ago

That is not niche at all. Pretty much any gov't, corp, etc that needs long term achives could do with faster access. 2 min is light speed since it's low(er) cost/decade/gb

Ekaros 2y ago

I see lot of reasonable use cases, say something like Internet Archive or library in general. Keep the frequented stuff in more hot cache and then rest of the stuff in colder storage. Without having to actually think about the process. Similar could be some new media production company. Archive B-roll or parts of older videos and then have the stuff you use very common available much faster.

lightedman 2y ago

This could be easily improved by keeping track of what's on the tape where in part of the SSD, like a length-based filesystem. Then it is just a matter of seeking to that length of tape and finding the actual start of the data. You could take the seek time for cold data to just a couple dozen seconds.