4 ms·

Long-term (decades), no-subscription archival storage. Essentially, you buy a block of space, upload your data over time, and it gets distributed when you need

by Hasz 3y ago

Long-term (decades), no-subscription archival storage. Essentially, you buy a block of space, upload your data over time, and it gets distributed when you need it (if you lost a primary backup), or on your death (to friends and relatives, or whoever you choose), or on a specific future date. It's a mashup of a safety deposit box, time capsule, and deadman switch. It's not ready yet, but will be ready in a few weeks. If you're interested, I would really like to talk to you. My email is ethan@ethanmye.rs FWIW, I did use chatgpt to write a lot of boilerplate JS and fix my bootstrap templates! edit deadpan->deadman

curtisblaine 3y ago
How are your clients sure that your storage lasts decades and doesn't end if you lose interest / fail / sell to another company? (They can't, of course, so: how do you convince them?)
- Hasz 3y ago
  It's a good question. A lot of it centers around creating good corporate governance and a reason for either me (or someone else) to stay interested (careful incentive design). This is obviously antithetical to the typical scale-up strategy of a tech company, and the financials are very similar to an insurance company. It's also why there is no free plan in the pricing. In terms of convincing, for the technically-minded, I have a public disaster recovery plan, a public business continuity plan, and "escape hatches" for "common" events -- war, subpoenas, changes in law, a post-RSA future, etc. The goal is to cover as many events as possible, including the very improbable (like AWS losing a whole AZ!). The backing clouds are Microsoft Files and AWS S3, both with excellent track records and absurdly good durability. There is some special caching in front to minimize cost. For the less technically-minded, there are no good alternatives. Self-hosted data is difficult to geographically distribute (and if you do it's difficult to update true cold storage). Cloud services have a very high lifetime cost, and unclear rules around data distribution to next of kin. Other methods, like burying some vacuum sealed MDISCs in a freezer, are not realistic. I am of the opinion that while it is impossible to predict the future, it is possible to plan for it.
  - NetOpWibby 3y ago
    That last sentence should be in your marketing
ethanwillis 3y ago
What's a deadpan switch?
- Hasz 3y ago
  oops, deadpan -> deadman https://en.wikipedia.org/wiki/Dead_man%27s_switch https://en.wikipedia.org/wiki/Dead_man%27s_switch It's not a perfect switch, but the idea is that you might have things you would only rather release when you're dead.
- mateo1 3y ago
  Maybe it's ChatGPT speaking, although would it be making such obvious mistakes?
elamje 3y ago
I worked on this in 2018/19. It's a really hard problem, and the only people I've found doing it at scale is a small Norwegian company called https://piql.no https://piql.no. They encode large files into film and store it in the Arctic World Archive - and they did GitHub's Arctic Code Vault. DNA storage is interesting, though can be damaged by radiation. Would love to see where you land with this project - it's one of the few cases where crypto may actually be the only real digital solution.
- Hasz 3y ago
  I think boring, standard solutions are actually pretty effective. We use commercial cloud cold storage for primary backup, with a self-hosted cache layer in front to drive down cost. Commercial off-the-shelf storage is also the best characterized and has a proven track record. NIST has some great reviews of the stability of optical media, and it’s quite good, done for the library of Congress. DNA storage would enable some pretty crazy storage density, but ensuring there’s a compatible reader around in 30 years might be difficult
EVa5I7bHFq9mnYK 3y ago
How it compares to the more established forever.com? I have an account but dont like that they dont accept zip/encrypted files. I have to diguise them as video files :)
- Hasz 3y ago
  Thanks for the link! It seems like they mostly focus on digitization, with some cloud storage being offered. A few key differences: Data distribution: one of the hard parts is actually sharing the memories you’ve saved when you die (or at some future date). When you create an archive, you choose someone to give that data to. Storage: no restrictions of what you upload, subject to US law. If you want to upload big encrypted blobs and give the key to someone else, I am happy to support it. Over the timespans we’re talking about, you want to look into quantum resistant encryption. Cost: on a per gb price, I am much cheaper, as there’s no digitization labor involved. I think digitization is declining, as even my grandma kept most of her pictures on her iPhone. There still exists a great digitization market, but it is getting smaller.

by Hasz 3y ago

curtisblaine 3y ago

How are your clients sure that your storage lasts decades and doesn't end if you lose interest / fail / sell to another company? (They can't, of course, so: how do you convince them?)

Hasz 3y ago
It's a good question. A lot of it centers around creating good corporate governance and a reason for either me (or someone else) to stay interested (careful incentive design). This is obviously antithetical to the typical scale-up strategy of a tech company, and the financials are very similar to an insurance company. It's also why there is no free plan in the pricing. In terms of convincing, for the technically-minded, I have a public disaster recovery plan, a public business continuity plan, and "escape hatches" for "common" events -- war, subpoenas, changes in law, a post-RSA future, etc. The goal is to cover as many events as possible, including the very improbable (like AWS losing a whole AZ!). The backing clouds are Microsoft Files and AWS S3, both with excellent track records and absurdly good durability. There is some special caching in front to minimize cost. For the less technically-minded, there are no good alternatives. Self-hosted data is difficult to geographically distribute (and if you do it's difficult to update true cold storage). Cloud services have a very high lifetime cost, and unclear rules around data distribution to next of kin. Other methods, like burying some vacuum sealed MDISCs in a freezer, are not realistic. I am of the opinion that while it is impossible to predict the future, it is possible to plan for it.
- NetOpWibby 3y ago
  That last sentence should be in your marketing

ethanwillis 3y ago

What's a deadpan switch?

Hasz 3y ago
oops, deadpan -> deadman https://en.wikipedia.org/wiki/Dead_man%27s_switch https://en.wikipedia.org/wiki/Dead_man%27s_switch It's not a perfect switch, but the idea is that you might have things you would only rather release when you're dead.
mateo1 3y ago
Maybe it's ChatGPT speaking, although would it be making such obvious mistakes?

elamje 3y ago

I worked on this in 2018/19. It's a really hard problem, and the only people I've found doing it at scale is a small Norwegian company called https://piql.no https://piql.no. They encode large files into film and store it in the Arctic World Archive - and they did GitHub's Arctic Code Vault. DNA storage is interesting, though can be damaged by radiation. Would love to see where you land with this project - it's one of the few cases where crypto may actually be the only real digital solution.

Hasz 3y ago
I think boring, standard solutions are actually pretty effective. We use commercial cloud cold storage for primary backup, with a self-hosted cache layer in front to drive down cost. Commercial off-the-shelf storage is also the best characterized and has a proven track record. NIST has some great reviews of the stability of optical media, and it’s quite good, done for the library of Congress. DNA storage would enable some pretty crazy storage density, but ensuring there’s a compatible reader around in 30 years might be difficult

EVa5I7bHFq9mnYK 3y ago

How it compares to the more established forever.com? I have an account but dont like that they dont accept zip/encrypted files. I have to diguise them as video files :)

Hasz 3y ago
Thanks for the link! It seems like they mostly focus on digitization, with some cloud storage being offered. A few key differences: Data distribution: one of the hard parts is actually sharing the memories you’ve saved when you die (or at some future date). When you create an archive, you choose someone to give that data to. Storage: no restrictions of what you upload, subject to US law. If you want to upload big encrypted blobs and give the key to someone else, I am happy to support it. Over the timespans we’re talking about, you want to look into quantum resistant encryption. Cost: on a per gb price, I am much cheaper, as there’s no digitization labor involved. I think digitization is declining, as even my grandma kept most of her pictures on her iPhone. There still exists a great digitization market, but it is getting smaller.