11 ms·

It's kinda mind-blowing that we have (so-called) AI, quantum computing, 6K screens, M2 NVME, billions of networked devices, etc., but regular data *can only be

It's kinda mind-blowing that we have (so-called) AI, quantum computing, 6K screens, M2 NVME, billions of networked devices, etc., but regular data *can only be expected to last about 5 years* due to the propensity of moving disk failure, SSD impermanence, bitrot, etc., and is only overcome with great attention and significant cost (continually maintaining a JBOD or RAID or NAS, or painstakingly burning to M-Disc bluray etc.) or handing it over to someone else to manage (cloud) or both. I mean maybe you get lucky with a simple 3-2-1 but maybe you don't, and for larger archives of data that is simply not necessarily a walk in the park either. Absolutely mindblowing.

pdimitar 2y ago
I share your disappointment. I explained it for myself with this: nobody cares if we the netizens have our data backed up. The corps want it for themselves and they face zero accountability if they lose it or share it illegally with others. So it's up to us really. I have a fairly OK setup, one copy on a local machine and several encrypted compressed copies in the cloud. It's not bulletproof but has saved my neck twice now, so can't complain. It's also manual... We the techies in general are dragging our feet on this though. We should have commoditized this stuff a decade ago because it's blindingly obvious the corps don't want to do it (not for free and with the quality we can do it anyway). Should have done app installers for all 3 major OS-es, zero-interaction-required unattended auto-updates -- make is to grandma would never know it's there and it's working. The only thing it asks is access to your cloud storage accounts and it decides automatically what goes where and how (kind of like disk RAID setups I suppose).
- keyringlight 2y ago
  I think in cases like this "personal computer" is a blessing and a curse. It seems like most of the big parties involved in computing with the big levers to move things are mostly in it for themselves and pick and choose whether they shoulder responsibility for a certain feature, or which side of 'personal' it comes down on with regards to if they hook it into their services and privacy. What would the attitudes be to losing physical information or property be for other parts of our lives versus digital info/property, if that was media, references, financial details, property deeds, old records, etc. While I do appreciate the generosity in how many projects make themselves available (free or otherwise), it does seem like they can have a narrow focus where they solve the challenge they had to solve, but aren't interested going past that point. There's logical reasons for why that happens, but there's unfulfilled potential to make personal computing a better environment there.
  - pdimitar 2y ago
    > it does seem like they can have a narrow focus where they solve the challenge they had to solve, but aren't interested going past that point. There's logical reasons for why that happens, but there's unfulfilled potential to make personal computing a better environment there. Oh absolutely, I agree so much with this. I get it, we can get obsessed with UNIX philosophy at times, one tool does its job perfectly etc. but we don't even make an attempt to assemble several things into one good cohesive whole. Sadly I am way too swamped with being out of a job and chronically ill but I just started becoming extremely bitter towards many more privileged techies who seem much more interested in farting out the 2489th LISP interpreter than solve a real world problem. :|
    - GTP 2y ago
      If you had the time and energy, how would your ideal solution look like?
      - pdimitar 2y ago
        Since we already have excellent tools that follow the UNIX philosophy then I would opt for being a LEGO assembly guy by combining these: - Management of backups themselves: borg / restic (or rustic) / duplicati / duplicacy etc. Last I evaluated them borg offered the most in one package. I like restic a bit more but it's a fair bit slower than borg, sadly. `rustic` aims to fix that but it's a volunteer effort and the author has bursts of productivity that are not on a stable schedule. I use rustic as a backup of my main backup tool (borg) but for now dare not use it exclusively. - Management of keys / secrets / passwords: my semi-vulnerable setup is to just upload my borg repo key to my Linux server and also have it pinned in one of my private Telegram channels. That one could be done better but I haven't done research. Probably something close to gpg (many modern alternatives with less friction but I forget names; I have them bookmarked though) to make sure even your keys are properly protected. One possibility is a password vault like keepass[x] or Enpass or others. - Management of storage: I use a local Linux server (EDIT: I have a very non-redundant and basic ZFS setup) and 5+ cloud storage services on the free tier. All of them have at least 5GB and my borg repository is barely 150MB I think. That includes ~20 past backups. The way these could be combined is mostly by having a top-notch GUI and CLI (so we serve different kinds of people) that allows for granting access to cloud storage servers, asks optionally for a local storage server (NFS, Samba, WebDAV etc.) and just does everything else by itself. It's very doable.
creer 2y ago
Well, professionally, tape is it - there is technology and it lasts more than 5 years. Unfortunately, the market for tape has evolved such that it's not very friendly to the non-pros. Not impossible but not friendly. That probably has to do with the lack of perceived market for that among non-corporate - or perhaps the impression that clown storage is where it's at for non-corporate. To be fair some more, JBOD/RAID and hard drives does work pretty well. Past the 5 year horizon to be sure. Product mgt and corp finance has also fallen in love with subscriptions - and clown storage is such an awesome match for that! Who needs to sell long term terabyte solutions when you can rent it out. Easy to argue against that logic of course, but not easy to fight.
- tombert 2y ago
  I have an LTO-6 tape drive. It works fine, but it is a pain in the ass to set up on Linux. It only connects via SAS, you need to load a lot of arcane kernel modules, the logs are non-standardized and often misleading, and the entire interface is command line based. I don't mind living in the command line and I don't even mind fighting to get everything up, but I don't see most people putting up with it. It's also a huge pain to get working with a laptop, since I don't think most laptops have a SAS connector, so you have to use an eGPU case with a Host Bus Adapter, which is its own share of headaches.
  - MisterTea 2y ago
    And here I just bought a NIB LTO-5 SAS drive for home file server backup.
  - creer 2y ago
    Hehe. If you're trying to live with just a laptop and an LTO drive, I can certainly see it! I'd expect most people who get into LTO drives have a massive set of hard drives, some convoluted desktop-and-up case(s) to run them, a couple laptops and random peripherals such as cameras, scanners and whatnot. USB all over the place. - So that for them both the drive and the command line interface are deeper in a technology map. But that goes back to the minimal market for LTO among amateurs. They will scratch their itch and write software for it but it's not exactly critical mass.
bob1029 2y ago
I think it's even more mind-blowing that we can hold back the tide of entropy for as long as we can and with so little energy expended. Solid state electronics and magnetic media are beyond magical. The odds of keeping terabytes of data on rails are astronomically bad.
Rygian 2y ago
Emphatic Yes. I'd like to expand. What I find mindblowing about it is that, as a regular consumer: * When you need more space you can't just plug in another disk or USB stick. You also have to choose on which device you want to use it, and you have to tell all your software to use it. And that may involve shuffling data around. * As a corollary, you need to remember in which device you put which stuff. * As an extra corollary, any data loss is catastrophic by default. * File copy operations still fail, and when they fail, they do so without ACID-strong commit/fallback semantics. * Backups don't happen by default, and are not transparent to the end user. * Data corruption can be silent. Bonus, but related: * You can't share arbitrary files with people without going through a 3rd party.
- nine_k 2y ago
  This is because most of the non-technical retail customers don't value reliability too much. They want more capacity and good speed at the lowest price point. Hence the proliferation of QLC / MLC NVMe drives with a small SLC write cache. It's not very stable, and it can only keep the write speed high for small files, but hey, you can get a terabyte for $50, and it loads the latest game real fast! Similarly, many users don't have much valuable, unique data on their computers. The most important stuff very often lives completely in the cloud, where it's properly backed up, etc. Also, the most ubiquitous computing device now is a smartphone. It has all the automatic backup stuff built in, you can put an SD card into it, and it will transparently extend the free space, without hassle. Even on PCs, MS and Apple nudge the users very prominently to use OneDrive and Apple Cloud for backing up their desktops / laptops. But past certain size, it costs money though, hence many people opt out of that. Again, because most people value the lowest price, and just hope for the best; because what could ever happen? Silent data corruption can still be an issue, but, frankly, malware is a much bigger threat for a typical non-technical user. Technical people have no trouble setting up all that: mount your disks under LVM, run ZFS on top of it, set up multiple backups, set up their own "magic wormhole" to share files with stranger easily. But they know why they are doing that. Educating users about IT hygiene is key for improving it, much like educating people about the dangers of not washing hands, or of eating unhealthy stuff, helped improve their health.
  - 2y ago
vodou 2y ago
How good/bad would it be to have a poor man's tape archival, using standard cassette tapes (C90, C120, etc)? For example, using something like ggwave [1]. I guess that would last way more than 5 years (although the data density is rather poor). [1] https://github.com/ggerganov/ggwave https://github.com/ggerganov/ggwave
- creer 2y ago
  LTO tape tech has gotten into pretty nutty territory - in order to achieve its density and speed. It wasn't "easy". So, so far away from C90 technology.
- dmd 2y ago
  A C120 can store, very generously, about 1 megabyte. A LTO8 tape stores 12,000,000 megabytes.
- EvanAnderson 2y ago
  > ... (although the data density is rather poor). "Rather poor" is putting it mildly. This sent me down a sort rabbit hole. From a Stack Exchange discussion[0] it was a short trip to exceedingly technical discussion about using QAM encoding[1] to really beef-up the storage capability. With the wacky QAM encoding tt looks like maybe 20MB per C90 cassette (and 90 minutes to "read" it back). [0] https://retrocomputing.stackexchange.com/questions/9260/how-much-data-could-a-home-computer-store-on-an-audio-cassette https://retrocomputing.stackexchange.com/questions/9260/how-... [1] https://redfrontdoor.org/blog/?p=795 https://redfrontdoor.org/blog/?p=795
  - rbanffy 2y ago
    It's interesting. I wouldn't dare to go beyond a 1500 baud signaling rate, but, then, audio tape is amenable to QAM, and that could multiply the transmission to 16 bits per token or more depending on the quality of the tape and recorder. I would be careful with that, however. If you are archiving your data, it's because you like it and, if you like your data, you want it to be readable a long time from now. I'd suggest vinyl records rather than tapes, as they are very robust, and can be read without physical contact.
remus 2y ago
In some ways it is surprising, but the examples you gave are only currently straightforward because of massive investment over many years by thousands of people. If you wanted to build chatGPT from scratch I'm sure it would be pretty hard, so it doesn't seem so unreasonable that you might pay someone if you care about keeping your data around for extended periods of time.
klysm 2y ago
Making stuff last for a long time is very difficult. It's cheaper to make things last for a short time and allow improvement
WalterBright 2y ago
I buy new disk drives every year, copy everything onto them, and retire the old ones as backups.
rietta 2y ago
Yes, it is. I actually have been mulling over a fictional world set in the future where the period between the 20th and 25th centuries is a mysterious time that so little is known about. The story follows a professor who is obsessed with the "Bit Rot Era" and finding out just what happened to that civilization. I have a prototype first chapter written that cold opens with an archeological dig '...John Li Wei looked up from his field journal where he had just written “No artifacts found in Basement Level 1, Site 46-012-0023”, wiping sweat from his brow. "Did you find something, Arnold?" he asked, his voice weary. "Three days in this godforsaken jungle, and we've got nothing but mud to show for it. Every book in this library’s long since turned to muck.” Arnold gestured towards the section of the site he had been laboring for the last 30 minutes, digging through layer after layer of brown muck, with fragments of metal hardware that once supported shelving. A glint of metal caught the filtered light. “Arnold, that’s just another computer case,” John sighed, his shoulders slumping slightly. He could already imagine the corroded metal and the disintegrated components inside. Useless. “Help me pull this out.” The two men strained against the clinging earth, their boots sinking into the mud with each heave. As they finally wrestled the heavy, corroded metal case free, a piercing shriek cut through the jungle sounds – beep, beep, beep, beep....'
- NetOpWibby 2y ago
  I'm riveted, please share more.
  - rietta 2y ago
    I suppose I can publish on by blog and then keep on writing. Can discover this future together along with anyone who wants to read it. This is my first serious attempt at fiction. I have only just begun. This is another portion of later in the first chapter. '...The transport, a ground-based vehicle, levitated silently outside. John glanced at his chronometer as he boarded. Jeg er ked af det, he thought, I'm late. The doors hissed shut, and the vehicle computer announced, “Destination: University. Estimated arrival: 25 minutes.” With a gentle hum, the vehicle glided smoothly along the elevated guideway. The air inside was cool and faintly metallic. Outside, the landscape was a patchwork of green fields, managed forests, and gleaming white research facilities. The transport’s progress was slow; the gentle sway and hum of the engines were a constant reminder of the NAU’s strict energy policies. John sighed, thinking about the upcoming lecture. How could he convey the importance of the Bit Rot Era when so many were focused on the pressing needs of the present? At the University, John rushed into the classroom, three students already waiting. “Jeg er ked af det, I’m late,” John said, a slight Danish accent coloring his Danglish. The classroom was intimate, a very different design from the great lecture halls of antiquity, designed for perhaps twenty students, with a central platform surrounded by holographic cameras for remote attendees. Historical maps and timelines adorned the interactive displays lining the walls. John quickly moved to the lectern and carefully removed six artifacts from his bag. “Velkommen to Ancient North America 1,” he began. “Welcome to Ancient North America 1. This is the class where you are going to learn about the past that was and the present that may yet be.” He gestured to a holographic timeline that appeared above the lectern. “In broad strokes, we consider the history of Ancient North America to consist of four periods: the pre-colonial, the rise of the nations – of Canada, the United States of America, and Mexico – the pre-Collapse, and the post-Collapse periods. You can take in-depth courses on most of these. Dr. Jones’ ‘Rise of the Nations’ is well worth your study to learn about the United States, its Constitution, and Canada. This very university is located in a place that was once called Newfoundland, Canada, and in ancient times the climate was quite harsh. Meget forskelligt from the lush green farmland, forests, and even nice beaches we have today. You can also take Dr. Pech’s history of the pre-colonial tribes and empires. That will teach you a great deal about the people who first inhabited the continent and learn about their history, culture, foodstuffs, many of which are still considered extinct. Desværre. “However,” John continued, his tone shifting, “the class you cannot take is for what we call the Bit Rot Era. A ‘bit’… imagine a light switch. On or off. That’s a bit. One or zero. The basic unit of digital information. The Bit Rot Era, from roughly the 20th to the 25th century, is a complete black box. En sort boks. What we do know comes from fragments of writing on paper. Millions of books were printed, but even those are often lost to time. We have fragments of ancient texts that talk about computers in every home and the ‘digitization’ of information and libraries. Digitization was the process of scanning physical media into computers. We’ve recovered millions of artifacts – disintegrated polycarbonates, silica, and bits of rare metals that were once these computers. But nothing within survived. Then something happened. The books vanished. The period from about the 21st to the 25th centuries was, as it were, expunged from history. Research indicates they went entirely digital, depending on system administrators to maintain the data… until the administrators stopped.”...'
mtillman 2y ago
Important to remember that M-Disc Blu-ray is marketing only and not spec whereas M-disc DVD is an actual spec for archival purposes.
UltraSane 2y ago
Data on LTO tapes stored correctly should last 40 years.
- sigio 2y ago
  Now you just need to find a drive that still works in 40 years, and can connect to a computer you have in 40 years. LTO drives will only read media from 1 generation off the specified version (LTO 9 will read LTO 8 tapes, not LTO-7) So you will need a 40 year old drive by then... good luck finding one that works.
  - UltraSane 2y ago
    If you really need to read data from a 40 year old tape you must also store a computer able to read the tape. Most companies with large LTO archives migrate tape from generation to generation, sometimes skipping one. That is why LTO drives can read/write back one gen while they can only read 2 gens back. BUT LTO is the most common tape format by a LOT so it is pretty likely that even 40 years from today there will be drives that can read LTO7,8,9
RGamma 2y ago
I often think we're living in a dark age (an age that is characterized by little surviving cultural output)... Dpends on how thing's will go, of course, but I ain't holding my breath.
- biofox 2y ago
  An important reason to continue buying printed books and supporting print media. Most web content I've consumed in my lifetime is already lost, many floppy disks and burned CD-ROMs I held onto are now unreadable, and in 200 years, the situation will only be worse. But I can go to the British Library and read a 1000+ year old text without much difficulty.
  - RGamma 2y ago
    FWIW does any of it remain on archive.org? They have some really great collections on there, and writing on digital archiving, like how to preserve CDs as best as possible (spoiler: the brand of CD is a large factor).
    - glenneroo 2y ago
      That's well and good for the _current_ ability to potentially find some old media, but regarding GP's point about digital data being fickle over long time-spans, Archive.org is only one hack/fire/solar-flare/DDOS/etc. away from being gone (potentially forever). They were recently hacked, thankfully the group who claimed responsibility weren't trying to destroy data. From: https://archive.org/about https://archive.org/about > A single copy of the Internet Archive library collection occupies 145+ Petabytes of server space -- and we store at least 2 copies of everything). Thankfully they have a few backups around the world, but there are still plenty of plausible events which could render the data useless. In contrast, physical mediums like books, vinyl records or microfilm/microfiche can last for hundreds of years.
  - account42 2y ago
    It's a mistake to assume that just because there are 1000 year old text that the optimized-for-cost prints you can buy today will also last that long.

pdimitar 2y ago

I share your disappointment. I explained it for myself with this: nobody cares if we the netizens have our data backed up. The corps want it for themselves and they face zero accountability if they lose it or share it illegally with others. So it's up to us really. I have a fairly OK setup, one copy on a local machine and several encrypted compressed copies in the cloud. It's not bulletproof but has saved my neck twice now, so can't complain. It's also manual... We the techies in general are dragging our feet on this though. We should have commoditized this stuff a decade ago because it's blindingly obvious the corps don't want to do it (not for free and with the quality we can do it anyway). Should have done app installers for all 3 major OS-es, zero-interaction-required unattended auto-updates -- make is to grandma would never know it's there and it's working. The only thing it asks is access to your cloud storage accounts and it decides automatically what goes where and how (kind of like disk RAID setups I suppose).

keyringlight 2y ago
I think in cases like this "personal computer" is a blessing and a curse. It seems like most of the big parties involved in computing with the big levers to move things are mostly in it for themselves and pick and choose whether they shoulder responsibility for a certain feature, or which side of 'personal' it comes down on with regards to if they hook it into their services and privacy. What would the attitudes be to losing physical information or property be for other parts of our lives versus digital info/property, if that was media, references, financial details, property deeds, old records, etc. While I do appreciate the generosity in how many projects make themselves available (free or otherwise), it does seem like they can have a narrow focus where they solve the challenge they had to solve, but aren't interested going past that point. There's logical reasons for why that happens, but there's unfulfilled potential to make personal computing a better environment there.
- pdimitar 2y ago
  > it does seem like they can have a narrow focus where they solve the challenge they had to solve, but aren't interested going past that point. There's logical reasons for why that happens, but there's unfulfilled potential to make personal computing a better environment there. Oh absolutely, I agree so much with this. I get it, we can get obsessed with UNIX philosophy at times, one tool does its job perfectly etc. but we don't even make an attempt to assemble several things into one good cohesive whole. Sadly I am way too swamped with being out of a job and chronically ill but I just started becoming extremely bitter towards many more privileged techies who seem much more interested in farting out the 2489th LISP interpreter than solve a real world problem. :|
  - GTP 2y ago
    If you had the time and energy, how would your ideal solution look like?
    - pdimitar 2y ago
      Since we already have excellent tools that follow the UNIX philosophy then I would opt for being a LEGO assembly guy by combining these: - Management of backups themselves: borg / restic (or rustic) / duplicati / duplicacy etc. Last I evaluated them borg offered the most in one package. I like restic a bit more but it's a fair bit slower than borg, sadly. `rustic` aims to fix that but it's a volunteer effort and the author has bursts of productivity that are not on a stable schedule. I use rustic as a backup of my main backup tool (borg) but for now dare not use it exclusively. - Management of keys / secrets / passwords: my semi-vulnerable setup is to just upload my borg repo key to my Linux server and also have it pinned in one of my private Telegram channels. That one could be done better but I haven't done research. Probably something close to gpg (many modern alternatives with less friction but I forget names; I have them bookmarked though) to make sure even your keys are properly protected. One possibility is a password vault like keepass[x] or Enpass or others. - Management of storage: I use a local Linux server (EDIT: I have a very non-redundant and basic ZFS setup) and 5+ cloud storage services on the free tier. All of them have at least 5GB and my borg repository is barely 150MB I think. That includes ~20 past backups. The way these could be combined is mostly by having a top-notch GUI and CLI (so we serve different kinds of people) that allows for granting access to cloud storage servers, asks optionally for a local storage server (NFS, Samba, WebDAV etc.) and just does everything else by itself. It's very doable.

creer 2y ago

Well, professionally, tape is it - there is technology and it lasts more than 5 years. Unfortunately, the market for tape has evolved such that it's not very friendly to the non-pros. Not impossible but not friendly. That probably has to do with the lack of perceived market for that among non-corporate - or perhaps the impression that clown storage is where it's at for non-corporate. To be fair some more, JBOD/RAID and hard drives does work pretty well. Past the 5 year horizon to be sure. Product mgt and corp finance has also fallen in love with subscriptions - and clown storage is such an awesome match for that! Who needs to sell long term terabyte solutions when you can rent it out. Easy to argue against that logic of course, but not easy to fight.

tombert 2y ago
I have an LTO-6 tape drive. It works fine, but it is a pain in the ass to set up on Linux. It only connects via SAS, you need to load a lot of arcane kernel modules, the logs are non-standardized and often misleading, and the entire interface is command line based. I don't mind living in the command line and I don't even mind fighting to get everything up, but I don't see most people putting up with it. It's also a huge pain to get working with a laptop, since I don't think most laptops have a SAS connector, so you have to use an eGPU case with a Host Bus Adapter, which is its own share of headaches.
- MisterTea 2y ago
  And here I just bought a NIB LTO-5 SAS drive for home file server backup.
- creer 2y ago
  Hehe. If you're trying to live with just a laptop and an LTO drive, I can certainly see it! I'd expect most people who get into LTO drives have a massive set of hard drives, some convoluted desktop-and-up case(s) to run them, a couple laptops and random peripherals such as cameras, scanners and whatnot. USB all over the place. - So that for them both the drive and the command line interface are deeper in a technology map. But that goes back to the minimal market for LTO among amateurs. They will scratch their itch and write software for it but it's not exactly critical mass.

bob1029 2y ago

I think it's even more mind-blowing that we can hold back the tide of entropy for as long as we can and with so little energy expended. Solid state electronics and magnetic media are beyond magical. The odds of keeping terabytes of data on rails are astronomically bad.

Rygian 2y ago

Emphatic Yes. I'd like to expand. What I find mindblowing about it is that, as a regular consumer: * When you need more space you can't just plug in another disk or USB stick. You also have to choose on which device you want to use it, and you have to tell all your software to use it. And that may involve shuffling data around. * As a corollary, you need to remember in which device you put which stuff. * As an extra corollary, any data loss is catastrophic by default. * File copy operations still fail, and when they fail, they do so without ACID-strong commit/fallback semantics. * Backups don't happen by default, and are not transparent to the end user. * Data corruption can be silent. Bonus, but related: * You can't share arbitrary files with people without going through a 3rd party.

nine_k 2y ago
This is because most of the non-technical retail customers don't value reliability too much. They want more capacity and good speed at the lowest price point. Hence the proliferation of QLC / MLC NVMe drives with a small SLC write cache. It's not very stable, and it can only keep the write speed high for small files, but hey, you can get a terabyte for $50, and it loads the latest game real fast! Similarly, many users don't have much valuable, unique data on their computers. The most important stuff very often lives completely in the cloud, where it's properly backed up, etc. Also, the most ubiquitous computing device now is a smartphone. It has all the automatic backup stuff built in, you can put an SD card into it, and it will transparently extend the free space, without hassle. Even on PCs, MS and Apple nudge the users very prominently to use OneDrive and Apple Cloud for backing up their desktops / laptops. But past certain size, it costs money though, hence many people opt out of that. Again, because most people value the lowest price, and just hope for the best; because what could ever happen? Silent data corruption can still be an issue, but, frankly, malware is a much bigger threat for a typical non-technical user. Technical people have no trouble setting up all that: mount your disks under LVM, run ZFS on top of it, set up multiple backups, set up their own "magic wormhole" to share files with stranger easily. But they know why they are doing that. Educating users about IT hygiene is key for improving it, much like educating people about the dangers of not washing hands, or of eating unhealthy stuff, helped improve their health.
- 2y ago

vodou 2y ago

How good/bad would it be to have a poor man's tape archival, using standard cassette tapes (C90, C120, etc)? For example, using something like ggwave [1]. I guess that would last way more than 5 years (although the data density is rather poor). [1] https://github.com/ggerganov/ggwave https://github.com/ggerganov/ggwave

creer 2y ago
LTO tape tech has gotten into pretty nutty territory - in order to achieve its density and speed. It wasn't "easy". So, so far away from C90 technology.
dmd 2y ago
A C120 can store, very generously, about 1 megabyte. A LTO8 tape stores 12,000,000 megabytes.
EvanAnderson 2y ago
> ... (although the data density is rather poor). "Rather poor" is putting it mildly. This sent me down a sort rabbit hole. From a Stack Exchange discussion[0] it was a short trip to exceedingly technical discussion about using QAM encoding[1] to really beef-up the storage capability. With the wacky QAM encoding tt looks like maybe 20MB per C90 cassette (and 90 minutes to "read" it back). [0] https://retrocomputing.stackexchange.com/questions/9260/how-much-data-could-a-home-computer-store-on-an-audio-cassette https://retrocomputing.stackexchange.com/questions/9260/how-... [1] https://redfrontdoor.org/blog/?p=795 https://redfrontdoor.org/blog/?p=795
- rbanffy 2y ago
  It's interesting. I wouldn't dare to go beyond a 1500 baud signaling rate, but, then, audio tape is amenable to QAM, and that could multiply the transmission to 16 bits per token or more depending on the quality of the tape and recorder. I would be careful with that, however. If you are archiving your data, it's because you like it and, if you like your data, you want it to be readable a long time from now. I'd suggest vinyl records rather than tapes, as they are very robust, and can be read without physical contact.

remus 2y ago

In some ways it is surprising, but the examples you gave are only currently straightforward because of massive investment over many years by thousands of people. If you wanted to build chatGPT from scratch I'm sure it would be pretty hard, so it doesn't seem so unreasonable that you might pay someone if you care about keeping your data around for extended periods of time.

klysm 2y ago

Making stuff last for a long time is very difficult. It's cheaper to make things last for a short time and allow improvement

WalterBright 2y ago

I buy new disk drives every year, copy everything onto them, and retire the old ones as backups.

rietta 2y ago

Yes, it is. I actually have been mulling over a fictional world set in the future where the period between the 20th and 25th centuries is a mysterious time that so little is known about. The story follows a professor who is obsessed with the "Bit Rot Era" and finding out just what happened to that civilization. I have a prototype first chapter written that cold opens with an archeological dig '...John Li Wei looked up from his field journal where he had just written “No artifacts found in Basement Level 1, Site 46-012-0023”, wiping sweat from his brow. "Did you find something, Arnold?" he asked, his voice weary. "Three days in this godforsaken jungle, and we've got nothing but mud to show for it. Every book in this library’s long since turned to muck.” Arnold gestured towards the section of the site he had been laboring for the last 30 minutes, digging through layer after layer of brown muck, with fragments of metal hardware that once supported shelving. A glint of metal caught the filtered light. “Arnold, that’s just another computer case,” John sighed, his shoulders slumping slightly. He could already imagine the corroded metal and the disintegrated components inside. Useless. “Help me pull this out.” The two men strained against the clinging earth, their boots sinking into the mud with each heave. As they finally wrestled the heavy, corroded metal case free, a piercing shriek cut through the jungle sounds – beep, beep, beep, beep....'

NetOpWibby 2y ago
I'm riveted, please share more.
- rietta 2y ago
  I suppose I can publish on by blog and then keep on writing. Can discover this future together along with anyone who wants to read it. This is my first serious attempt at fiction. I have only just begun. This is another portion of later in the first chapter. '...The transport, a ground-based vehicle, levitated silently outside. John glanced at his chronometer as he boarded. Jeg er ked af det, he thought, I'm late. The doors hissed shut, and the vehicle computer announced, “Destination: University. Estimated arrival: 25 minutes.” With a gentle hum, the vehicle glided smoothly along the elevated guideway. The air inside was cool and faintly metallic. Outside, the landscape was a patchwork of green fields, managed forests, and gleaming white research facilities. The transport’s progress was slow; the gentle sway and hum of the engines were a constant reminder of the NAU’s strict energy policies. John sighed, thinking about the upcoming lecture. How could he convey the importance of the Bit Rot Era when so many were focused on the pressing needs of the present? At the University, John rushed into the classroom, three students already waiting. “Jeg er ked af det, I’m late,” John said, a slight Danish accent coloring his Danglish. The classroom was intimate, a very different design from the great lecture halls of antiquity, designed for perhaps twenty students, with a central platform surrounded by holographic cameras for remote attendees. Historical maps and timelines adorned the interactive displays lining the walls. John quickly moved to the lectern and carefully removed six artifacts from his bag. “Velkommen to Ancient North America 1,” he began. “Welcome to Ancient North America 1. This is the class where you are going to learn about the past that was and the present that may yet be.” He gestured to a holographic timeline that appeared above the lectern. “In broad strokes, we consider the history of Ancient North America to consist of four periods: the pre-colonial, the rise of the nations – of Canada, the United States of America, and Mexico – the pre-Collapse, and the post-Collapse periods. You can take in-depth courses on most of these. Dr. Jones’ ‘Rise of the Nations’ is well worth your study to learn about the United States, its Constitution, and Canada. This very university is located in a place that was once called Newfoundland, Canada, and in ancient times the climate was quite harsh. Meget forskelligt from the lush green farmland, forests, and even nice beaches we have today. You can also take Dr. Pech’s history of the pre-colonial tribes and empires. That will teach you a great deal about the people who first inhabited the continent and learn about their history, culture, foodstuffs, many of which are still considered extinct. Desværre. “However,” John continued, his tone shifting, “the class you cannot take is for what we call the Bit Rot Era. A ‘bit’… imagine a light switch. On or off. That’s a bit. One or zero. The basic unit of digital information. The Bit Rot Era, from roughly the 20th to the 25th century, is a complete black box. En sort boks. What we do know comes from fragments of writing on paper. Millions of books were printed, but even those are often lost to time. We have fragments of ancient texts that talk about computers in every home and the ‘digitization’ of information and libraries. Digitization was the process of scanning physical media into computers. We’ve recovered millions of artifacts – disintegrated polycarbonates, silica, and bits of rare metals that were once these computers. But nothing within survived. Then something happened. The books vanished. The period from about the 21st to the 25th centuries was, as it were, expunged from history. Research indicates they went entirely digital, depending on system administrators to maintain the data… until the administrators stopped.”...'

mtillman 2y ago

Important to remember that M-Disc Blu-ray is marketing only and not spec whereas M-disc DVD is an actual spec for archival purposes.

UltraSane 2y ago

Data on LTO tapes stored correctly should last 40 years.

sigio 2y ago
Now you just need to find a drive that still works in 40 years, and can connect to a computer you have in 40 years. LTO drives will only read media from 1 generation off the specified version (LTO 9 will read LTO 8 tapes, not LTO-7) So you will need a 40 year old drive by then... good luck finding one that works.
- UltraSane 2y ago
  If you really need to read data from a 40 year old tape you must also store a computer able to read the tape. Most companies with large LTO archives migrate tape from generation to generation, sometimes skipping one. That is why LTO drives can read/write back one gen while they can only read 2 gens back. BUT LTO is the most common tape format by a LOT so it is pretty likely that even 40 years from today there will be drives that can read LTO7,8,9

RGamma 2y ago

I often think we're living in a dark age (an age that is characterized by little surviving cultural output)... Dpends on how thing's will go, of course, but I ain't holding my breath.

biofox 2y ago
An important reason to continue buying printed books and supporting print media. Most web content I've consumed in my lifetime is already lost, many floppy disks and burned CD-ROMs I held onto are now unreadable, and in 200 years, the situation will only be worse. But I can go to the British Library and read a 1000+ year old text without much difficulty.
- RGamma 2y ago
  FWIW does any of it remain on archive.org? They have some really great collections on there, and writing on digital archiving, like how to preserve CDs as best as possible (spoiler: the brand of CD is a large factor).
  - glenneroo 2y ago
    That's well and good for the _current_ ability to potentially find some old media, but regarding GP's point about digital data being fickle over long time-spans, Archive.org is only one hack/fire/solar-flare/DDOS/etc. away from being gone (potentially forever). They were recently hacked, thankfully the group who claimed responsibility weren't trying to destroy data. From: https://archive.org/about https://archive.org/about > A single copy of the Internet Archive library collection occupies 145+ Petabytes of server space -- and we store at least 2 copies of everything). Thankfully they have a few backups around the world, but there are still plenty of plausible events which could render the data useless. In contrast, physical mediums like books, vinyl records or microfilm/microfiche can last for hundreds of years.
- account42 2y ago
  It's a mistake to assume that just because there are 1000 year old text that the optimized-for-cost prints you can buy today will also last that long.