5 ms·

IMO the concept of long-term bulk storage doesn't really make sense: Most of your stuff will be recycled by your grandkids. While you're alive, you can just bu

by NavinF 2y ago

IMO the concept of long-term bulk storage doesn't really make sense: Most of your stuff will be recycled by your grandkids. While you're alive, you can just buy a couple of NAS, stash one at a friend's/parent's place, buy the cheapest $/TB storage, use raidz to automatically correct errors over time, and set it up to email you every time it had to resiver to a hot spare so you can buy another HDD >only a fool would trust them to last Heh, head over to /r/DataHoarder to see more of that. "but this time it's different, my coasters don't use organic dyes!"

lencastre 2y ago
Tell me more about raidz and resiver…
- NavinF 2y ago
  Hm? It automatically replaces the dead drive with a hot spare over a couple of days. You can also do it in a couple of hours with dRAID: https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAID%20Howto.html https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAI... You just need a couple of parity drives per vdev and one hot spare for the whole array
eru 2y ago
> Most of your stuff will be recycled by your grandkids. Could you explain this? Why would they do that?
- NavinF 2y ago
  - Most people are not as interesting as they think they are - It's unlikely that your grandkids would have the same interests as you - If anything in your NAS was relevant to the family, your SO would have kept a copy of it in her icloud instead of using your self-hosted photo viewer over tailscale - Once you're old you almost certainly won't be using modern software and file formats. Accessing your data will be incredibly inconvenient
  - sdwr 2y ago
    Or, written even less kind, the intersection of sets of: - people with interesting lives - people who meticulously archive and backup all their personal data is close to 0. You're either doing or you're preserving, so if you preserve what you do, you aren't doing anything worth preserving.
    - eru 2y ago
      It's very, very hard to preserve only exactly what you do, and nothing else. There's always context, and people might be interested in the context, even if they ain't interested in you. Btw, there's lots of diaries and autobiographies published. Some of them quite popular.
  - shiroiushi 2y ago
    >Once you're old you almost certainly won't be using modern software and file formats. Accessing your data will be incredibly inconvenient This isn't true. Sure, no one uses WMV or ZOO these days, but you can still get tools to read them. But those were also not-so-popular formats/codecs that were replaced quickly by better stuff. MP3 is also old, but still very ubiquitous. Furthermore, the specs and software for modern file formats (like audio/video codecs) are all publicly available. People will still be able to read h.264 videos 50 years from now, don't worry. It's not going to be like the Domesday Book.
    - justsomehnguy 2y ago
      > But those were also not-so-popular formats/codecs that were replaced quickly by better stuff. Yes, but nobody said "let's get back and reencode everything in a new, better format"
    - eru 2y ago
      Yes, it's actually incredibly easy, if not always convenient, to read most of these old formats.
  - eru 2y ago
    > - Most people are not as interesting as they think they are > - It's unlikely that your grandkids would have the same interests as you The bar is pretty low: your grandkids only need to be interested enough to keep some tiny amounts of data around. (Assuming that data capacities keep growing, your perhaps dozen of TiB of data will fit on a thumb drive in the future, or perhaps even an email attachment.) Also your descendants don't need to find your data interesting for the same reasons you do. You might snap some pictures of your travels to famous landmarks (which your grand kids don't care about, because they can't find much better photos of the Eiffel Tower online), but they might be interested in how fashion changed over time, or weight or smartphones or whatever is in the background.
- jwagenet 2y ago
  Most people have no use for the things their parents or grandparents found sentimental or worth keeping, so they toss it.
  - shiroiushi 2y ago
    This is true, but depending on what the data is, hopefully that stuff could be donated to a library if the stuff lasted long enough to go into the public domain. Of course, if it's family videos, no one's going to care about that crap, but if it's pirated movies or whatever, some of that stuff might not be available otherwise. Just look at all the classic games from the 70s-80s that would be gone forever if people hadn't copied and distributed them.
    - jwagenet 2y ago
      Based on the whole recent VCF ordeal: https://news.ycombinator.com/item?id=40005150 https://news.ycombinator.com/item?id=40005150 , donating any of this crap to a library or anyone else and it actually being used is a pipe dream. These organizations don’t have the time or space to process anything, assuming they don’t have it anyway. Likely only the now deceased owner knew that they had xyz special material. Their kin processing their attic or garage likely don’t.
      - eru 2y ago
        Those were physical artefacts. They are a lot harder to handle, and don't get any smaller as time progresses. Here we are talking about data.
    - eru 2y ago
      Why would people not care about family photos? All the stuff in the background is super useful for historians. Keep in mind: old data is tiny by contemporary standards. So your grandchildren can just take the few TiB in your personal collection, and stick it on a futuristic thumb-drive somewhere. Barely takes up any space.
      - 2y ago
clan 2y ago
And you need to keep the bits on current technology. That is the real gambit. So your NAS idea is probably the best with the caveat at your need to upgrade it regularly as well. Compare it to a floppy disc from your grandparents. Not that bad for 3,5"? How about 5,25"? No? Then 8". These are now hard to come by. And this is just in the time span of 30-50 years. The best bulk storage format was actually available early on and very shelf stable if treated reasonably: Paper tape. Low density and readers are hard to come by today. Your primary point that our grandkids won't care is on point. We would drown if we keep everything. But the counter point would be that if we do not even try to preserve anything we would probably end up with nothing. But for every average Joe who does not care we seem to have plenty data hoarders to make up for that.
- nextos 2y ago
  I disagree a bit. Some things might require long-term data storage. For example, GPG keys, personal wallets, private documents and certificates. Imagine how much of your life can be wiped out by a Carrington Event, which is not that unlikely. I don't trust clouds for this. Right now, archival-grade DVDs are not a bad option. Almost 5 GB, and under good storage conditions these can last >50 years, probably more.
  - gond 2y ago
    Have you considered 100GB BD-RE? These are rewritable, but that is only the byproduct of being a phase-change medium, which is also >50 years and thus has a longer shelf life than the standard WORM BD/DVD. I am using them and can say only good things so far.

by NavinF 2y ago

lencastre 2y ago

Tell me more about raidz and resiver…

NavinF 2y ago
Hm? It automatically replaces the dead drive with a hot spare over a couple of days. You can also do it in a couple of hours with dRAID: https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAID%20Howto.html https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAI... You just need a couple of parity drives per vdev and one hot spare for the whole array

eru 2y ago

> Most of your stuff will be recycled by your grandkids. Could you explain this? Why would they do that?

NavinF 2y ago
- Most people are not as interesting as they think they are - It's unlikely that your grandkids would have the same interests as you - If anything in your NAS was relevant to the family, your SO would have kept a copy of it in her icloud instead of using your self-hosted photo viewer over tailscale - Once you're old you almost certainly won't be using modern software and file formats. Accessing your data will be incredibly inconvenient
- sdwr 2y ago
  Or, written even less kind, the intersection of sets of: - people with interesting lives - people who meticulously archive and backup all their personal data is close to 0. You're either doing or you're preserving, so if you preserve what you do, you aren't doing anything worth preserving.
  - eru 2y ago
    It's very, very hard to preserve only exactly what you do, and nothing else. There's always context, and people might be interested in the context, even if they ain't interested in you. Btw, there's lots of diaries and autobiographies published. Some of them quite popular.
- shiroiushi 2y ago
  >Once you're old you almost certainly won't be using modern software and file formats. Accessing your data will be incredibly inconvenient This isn't true. Sure, no one uses WMV or ZOO these days, but you can still get tools to read them. But those were also not-so-popular formats/codecs that were replaced quickly by better stuff. MP3 is also old, but still very ubiquitous. Furthermore, the specs and software for modern file formats (like audio/video codecs) are all publicly available. People will still be able to read h.264 videos 50 years from now, don't worry. It's not going to be like the Domesday Book.
  - justsomehnguy 2y ago
    > But those were also not-so-popular formats/codecs that were replaced quickly by better stuff. Yes, but nobody said "let's get back and reencode everything in a new, better format"
  - eru 2y ago
    Yes, it's actually incredibly easy, if not always convenient, to read most of these old formats.
- eru 2y ago
  > - Most people are not as interesting as they think they are > - It's unlikely that your grandkids would have the same interests as you The bar is pretty low: your grandkids only need to be interested enough to keep some tiny amounts of data around. (Assuming that data capacities keep growing, your perhaps dozen of TiB of data will fit on a thumb drive in the future, or perhaps even an email attachment.) Also your descendants don't need to find your data interesting for the same reasons you do. You might snap some pictures of your travels to famous landmarks (which your grand kids don't care about, because they can't find much better photos of the Eiffel Tower online), but they might be interested in how fashion changed over time, or weight or smartphones or whatever is in the background.
jwagenet 2y ago
Most people have no use for the things their parents or grandparents found sentimental or worth keeping, so they toss it.
- shiroiushi 2y ago
  This is true, but depending on what the data is, hopefully that stuff could be donated to a library if the stuff lasted long enough to go into the public domain. Of course, if it's family videos, no one's going to care about that crap, but if it's pirated movies or whatever, some of that stuff might not be available otherwise. Just look at all the classic games from the 70s-80s that would be gone forever if people hadn't copied and distributed them.
  - jwagenet 2y ago
    Based on the whole recent VCF ordeal: https://news.ycombinator.com/item?id=40005150 https://news.ycombinator.com/item?id=40005150 , donating any of this crap to a library or anyone else and it actually being used is a pipe dream. These organizations don’t have the time or space to process anything, assuming they don’t have it anyway. Likely only the now deceased owner knew that they had xyz special material. Their kin processing their attic or garage likely don’t.
    - eru 2y ago
      Those were physical artefacts. They are a lot harder to handle, and don't get any smaller as time progresses. Here we are talking about data.
  - eru 2y ago
    Why would people not care about family photos? All the stuff in the background is super useful for historians. Keep in mind: old data is tiny by contemporary standards. So your grandchildren can just take the few TiB in your personal collection, and stick it on a futuristic thumb-drive somewhere. Barely takes up any space.
    - 2y ago

clan 2y ago

And you need to keep the bits on current technology. That is the real gambit. So your NAS idea is probably the best with the caveat at your need to upgrade it regularly as well. Compare it to a floppy disc from your grandparents. Not that bad for 3,5"? How about 5,25"? No? Then 8". These are now hard to come by. And this is just in the time span of 30-50 years. The best bulk storage format was actually available early on and very shelf stable if treated reasonably: Paper tape. Low density and readers are hard to come by today. Your primary point that our grandkids won't care is on point. We would drown if we keep everything. But the counter point would be that if we do not even try to preserve anything we would probably end up with nothing. But for every average Joe who does not care we seem to have plenty data hoarders to make up for that.

nextos 2y ago
I disagree a bit. Some things might require long-term data storage. For example, GPG keys, personal wallets, private documents and certificates. Imagine how much of your life can be wiped out by a Carrington Event, which is not that unlikely. I don't trust clouds for this. Right now, archival-grade DVDs are not a bad option. Almost 5 GB, and under good storage conditions these can last >50 years, probably more.
- gond 2y ago
  Have you considered 100GB BD-RE? These are rewritable, but that is only the byproduct of being a phase-change medium, which is also >50 years and thus has a longer shelf life than the standard WORM BD/DVD. I am using them and can say only good things so far.