5 ms·

The Digital Dark Ages

74 points by zdw 4y ago

Havoc 4y ago
I'd say this should be seen in the context of quantity of information produced too though. Even if 90% is encrypted and lost it seems likely to me that the remaining 10% is so much in sheer volume that future historians will have little trouble piecing it all together. At least compared to say roman times where you're lucky to get anything at all
- slondr 4y ago
  On the flipside it seems like we could really easily end up in a situation like the whole thing about dinosaurs having feathers. If the 10% of surviving information isn't a representative sample of the other 90%, the conclusions future historians will draw will be very wrong.
tekchip 4y ago
I actually had the idea to help prevent this quite some time ago but unfortunately lack the skills to bring it to fruition. Maybe someone else can if I put it out there. Likely not, but here goes. It's rare to impossible for a single institution to maintain all such personal record. The internet archive does a valiant job, but barely scrapes the surface. The only way for this to work is if individuals maintain their own historical record. So the question is how? I figure most people aren't savvy so a company who provides a storage device, and perhaps services to help gather said data, would be essential. I'm thinking it works like life insurance where someone pays over time and their data is then delivered to them, at the conclusion of any payments, or to their family in the event of a death. Perhaps this could simply be rolled into life insurance or something of the like? That's the basic gist. Put the power to save this information in the hands of the people to maintain going forward. Like having someone's written notebook kept long into the future. Except it's a storage drive. Tech hurdles there for sure, but with things like MS's Project Silica, and the like, perhaps not insurmountable.
- woojoo666 4y ago
  The two solutions I've seen for this are Dead Mans Switch [1] and Shamir's Secret Sharing [2]. There's an interesting related discussion on StackExchange too [3] [1]: https://www.deadmansswitch.net/ https://www.deadmansswitch.net/ [2]: https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing [3]: https://security.stackexchange.com/questions/239111/how-to-share-passwords-only-after-death https://security.stackexchange.com/questions/239111/how-to-s...
- Jill_the_Pill 4y ago
  >> "The only way for this to work is if individuals maintain their own historical record." Why's that? We have long had effective institutional preservation of knowledge and historical records by public and university libraries. Why shift to the inefficiency of individual, amateur preservation rather than support skilled professionals to do it right? Right now, I am having a hard time tracking down a 2002 or 2003 CD-ROM (New_LocClim) that may have contained the weather station data underlying the FAO/GPCC rainfall product. It seems far more likely that a library or other institution, rather than some random person, would have preserved that information in a way I can discover and access.
BirAdam 4y ago
Using networked things as an example of why this a “dark age” is inaccurate. As long as the server and client software survive, the curious will make instances of both. This has already happened with large networked installations of other computing systems requiring clients and at least one server.
- rcoder 4y ago
  Which "server", exactly, will allow me to recover the messages from a conversation on Facebook after the service as we know it is long gone? (Out of business, moved entirely to the "metaverse", transcended along with the rest of the AI resistance, whatever.) Ditto my Twitter photo uploads, comment threads in GDocs, or app wireframes in Figma, much less code running on a cloud server somewhere. Once upon a time, by recording a thing you made it durable *by default* and even got a copy you could do with what you liked. Now, recording and sharing words or ideas comes with zero assumption of durability. Your service provider's ToS and business fortunes might support them keeping it around and accessible for now, but there is zero guarantee that will outlast the whims and fortunes of a SaaS providers. The combination of DRM (to lock down devices) and "everything lives in the cloud" (to keep custodianship and control of data in vendors' hands, not creators') means that without very intentional, often obscure and/or annoying manual steps, your correspondence, recordings, and other creative output will not survive you. Even _with_ painstaking effort there's every chance it won't work. Personally, I've tried for a couple of years now to get people I work with to write important things down somewhere that isn't Slack, GDocs or another pay-to-play service. Business, personal, silly, whatever: as it stands, all of that culture and history and creativity is just going to go "poof!" as soon as some company's C-suite does the math and decides the storage and compute to make it available don't pencil out any more. Now I just accept that unless I have 1) a complete, easily-cloned copy of something, 2) that I can access and manipulate using OSS software, 3) on a general-purpose computer I physically control, then that thing is not *mine*. If it isn't mine, I can't decide what happens to it in the near term, much less once I'm not around to lobby for access.
  - BirAdam 4y ago
    A lot is always lost. Nothing has changed. In this case, some amount of data will survive (especially because data is valuable as a commercial asset), the software itself will likely survive, and instances will later be setup in a virtualized environments. The preservation of data for practicality is very different from historical preservation. Most data from any era gets lost because people at the time do not consider historical value. They consider what is needed for their own goals. Beyond that, most books deteriorate, film deteriorates, etc this is no different from any of that. I understand and sympathize with the idea of open standards, open formats, and the desire for more accessible media and storage. I really do. The loss of ownership over everything is a bummer. However, pretending that we are more at risk of data loss now than before is not valid. DRM always gets cracked. Pirates always find a way. Encryption is likely already compromised by intelligence agencies who slurp everything up in their own archives. Data hoarders are a thing. Mass duplication of data is a thing. People have made recordings and films showing interactions with all of our current systems. People have made all kinds of documentation, documentaries, tutorials, and other media regarding our modern culture, modern technologies, and so on. This is the single most well documented period in history. If anything, we’re doing better at this now than ever before. The real issue for future historians will be sifting through the amount of material.
asciiresort 4y ago
> Nostalgia is the least inclusive interaction with a museum of historical artefacts because you cannot teach anyone nostalgia: they either were there and remember the thing on display, or they do not. That’s not true. A not uncommon type of museum in Japan are the “Showa Era Museums”. I was not alive during this era, but I feel nostalgia nonetheless. For context, these museums usually capture the 50s-70s, Japan’s rapid economic growth feeding into consumerism, or maybe the other way around, and there are record players, pachinko machines, and a very distinctive, bright, neon filled aesthetic.
jcranmer 4y ago
The idea that the digital world is launching us into another "Dark Age" is probably overselling the issue. Let's look at how much actually survives from the pre-Dark Ages. The complete corpus of Classical Latin and Greek literature can be found in the Loeb Classical Library. Which easily fits on a couple of bookshelves in your study (note that the books consist of the original text and a translation on the facing page). With a couple more volumes, of the Corpus Inscriptionum Latinarum, we can also add to our collection every single identified piece of inscription, including even an entire volume on mileposts. Similarly thorough works will doubtless exist for all classical languages, even non-European ones like Mayan or Chinese. Now walk into a library, even the tiny library of a small town. You are surrounded by more written text than survives from likely not only Classical Antiquity but all of human history through 1500. Even in the digital age, we are still producing more written text than most of human history. Probably, we are preserving a larger fraction of texts than were preserved from the Classical era. It's telling that the complaints of the loss of preservation are of the kinds of ephemera which don't exist for most periods of history. (Also telling that the person complaining about this loss of preservation is a "senior Research Software Engineer" and not a historian or archivist or similar field that actually deals with preserving or working with preserved data.)
- dwheeler 4y ago
  > The complete corpus of Classical Latin and Greek literature can be found in the Loeb Classical Library. Well, no. The complete surviving corpus, sure, but almost all of those works have been lost, never mind those of earlier eras. A short summary: https://en.wikipedia.org/wiki/Lost_literary_work https://en.wikipedia.org/wiki/Lost_literary_work For a specific example: I was trying to understand some of the history of logic. I learned that that in ancient Greek times Stoic logic was widely used. What was it? That is hard to answer, because practically everything is lost now. We've had to reconstruct the Stoic logic axioms from fragmentary sources, that's how bad it is. We know there was more, but we don't know exactly what it was.
  - jcranmer 4y ago
    Yes, the qualifier of "surviving" is very important. Trying to look quickly for a hard number, it seems that the usual estimate (perhaps one study being widely reported rather than hard estimates) is about 5-10% of works surviving. In the modern day, I think we are doing a better of job of preservation and archival than existed in Classical times. Take something like Geocities--we had a major archival organization step in to preserve a record of what is effectively non-notable people writing about non-notable things. While I'm unsure if that particular archival record will exist in 2,000 years, I strongly suspect that there will exist some notable archive of non-notable people writing about non-notable things, the kind of record that historians today can only dream of having from those who existed 2,000 years prior to us.

74 points by zdw 4y ago

I'd say this should be seen in the context of quantity of information produced too though. Even if 90% is encrypted and lost it seems likely to me that the remaining 10% is so much in sheer volume that future historians will have little trouble piecing it all together. At least compared to say roman times where you're lucky to get anything at all

slondr 4y ago
On the flipside it seems like we could really easily end up in a situation like the whole thing about dinosaurs having feathers. If the 10% of surviving information isn't a representative sample of the other 90%, the conclusions future historians will draw will be very wrong.

tekchip 4y ago

I actually had the idea to help prevent this quite some time ago but unfortunately lack the skills to bring it to fruition. Maybe someone else can if I put it out there. Likely not, but here goes. It's rare to impossible for a single institution to maintain all such personal record. The internet archive does a valiant job, but barely scrapes the surface. The only way for this to work is if individuals maintain their own historical record. So the question is how? I figure most people aren't savvy so a company who provides a storage device, and perhaps services to help gather said data, would be essential. I'm thinking it works like life insurance where someone pays over time and their data is then delivered to them, at the conclusion of any payments, or to their family in the event of a death. Perhaps this could simply be rolled into life insurance or something of the like? That's the basic gist. Put the power to save this information in the hands of the people to maintain going forward. Like having someone's written notebook kept long into the future. Except it's a storage drive. Tech hurdles there for sure, but with things like MS's Project Silica, and the like, perhaps not insurmountable.

woojoo666 4y ago
The two solutions I've seen for this are Dead Mans Switch [1] and Shamir's Secret Sharing [2]. There's an interesting related discussion on StackExchange too [3] [1]: https://www.deadmansswitch.net/ https://www.deadmansswitch.net/ [2]: https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing [3]: https://security.stackexchange.com/questions/239111/how-to-share-passwords-only-after-death https://security.stackexchange.com/questions/239111/how-to-s...
Jill_the_Pill 4y ago
>> "The only way for this to work is if individuals maintain their own historical record." Why's that? We have long had effective institutional preservation of knowledge and historical records by public and university libraries. Why shift to the inefficiency of individual, amateur preservation rather than support skilled professionals to do it right? Right now, I am having a hard time tracking down a 2002 or 2003 CD-ROM (New_LocClim) that may have contained the weather station data underlying the FAO/GPCC rainfall product. It seems far more likely that a library or other institution, rather than some random person, would have preserved that information in a way I can discover and access.

BirAdam 4y ago

Using networked things as an example of why this a “dark age” is inaccurate. As long as the server and client software survive, the curious will make instances of both. This has already happened with large networked installations of other computing systems requiring clients and at least one server.

rcoder 4y ago
Which "server", exactly, will allow me to recover the messages from a conversation on Facebook after the service as we know it is long gone? (Out of business, moved entirely to the "metaverse", transcended along with the rest of the AI resistance, whatever.) Ditto my Twitter photo uploads, comment threads in GDocs, or app wireframes in Figma, much less code running on a cloud server somewhere. Once upon a time, by recording a thing you made it durable *by default* and even got a copy you could do with what you liked. Now, recording and sharing words or ideas comes with zero assumption of durability. Your service provider's ToS and business fortunes might support them keeping it around and accessible for now, but there is zero guarantee that will outlast the whims and fortunes of a SaaS providers. The combination of DRM (to lock down devices) and "everything lives in the cloud" (to keep custodianship and control of data in vendors' hands, not creators') means that without very intentional, often obscure and/or annoying manual steps, your correspondence, recordings, and other creative output will not survive you. Even _with_ painstaking effort there's every chance it won't work. Personally, I've tried for a couple of years now to get people I work with to write important things down somewhere that isn't Slack, GDocs or another pay-to-play service. Business, personal, silly, whatever: as it stands, all of that culture and history and creativity is just going to go "poof!" as soon as some company's C-suite does the math and decides the storage and compute to make it available don't pencil out any more. Now I just accept that unless I have 1) a complete, easily-cloned copy of something, 2) that I can access and manipulate using OSS software, 3) on a general-purpose computer I physically control, then that thing is not *mine*. If it isn't mine, I can't decide what happens to it in the near term, much less once I'm not around to lobby for access.
- BirAdam 4y ago
  A lot is always lost. Nothing has changed. In this case, some amount of data will survive (especially because data is valuable as a commercial asset), the software itself will likely survive, and instances will later be setup in a virtualized environments. The preservation of data for practicality is very different from historical preservation. Most data from any era gets lost because people at the time do not consider historical value. They consider what is needed for their own goals. Beyond that, most books deteriorate, film deteriorates, etc this is no different from any of that. I understand and sympathize with the idea of open standards, open formats, and the desire for more accessible media and storage. I really do. The loss of ownership over everything is a bummer. However, pretending that we are more at risk of data loss now than before is not valid. DRM always gets cracked. Pirates always find a way. Encryption is likely already compromised by intelligence agencies who slurp everything up in their own archives. Data hoarders are a thing. Mass duplication of data is a thing. People have made recordings and films showing interactions with all of our current systems. People have made all kinds of documentation, documentaries, tutorials, and other media regarding our modern culture, modern technologies, and so on. This is the single most well documented period in history. If anything, we’re doing better at this now than ever before. The real issue for future historians will be sifting through the amount of material.

asciiresort 4y ago

> Nostalgia is the least inclusive interaction with a museum of historical artefacts because you cannot teach anyone nostalgia: they either were there and remember the thing on display, or they do not. That’s not true. A not uncommon type of museum in Japan are the “Showa Era Museums”. I was not alive during this era, but I feel nostalgia nonetheless. For context, these museums usually capture the 50s-70s, Japan’s rapid economic growth feeding into consumerism, or maybe the other way around, and there are record players, pachinko machines, and a very distinctive, bright, neon filled aesthetic.

jcranmer 4y ago

The idea that the digital world is launching us into another "Dark Age" is probably overselling the issue. Let's look at how much actually survives from the pre-Dark Ages. The complete corpus of Classical Latin and Greek literature can be found in the Loeb Classical Library. Which easily fits on a couple of bookshelves in your study (note that the books consist of the original text and a translation on the facing page). With a couple more volumes, of the Corpus Inscriptionum Latinarum, we can also add to our collection every single identified piece of inscription, including even an entire volume on mileposts. Similarly thorough works will doubtless exist for all classical languages, even non-European ones like Mayan or Chinese. Now walk into a library, even the tiny library of a small town. You are surrounded by more written text than survives from likely not only Classical Antiquity but all of human history through 1500. Even in the digital age, we are still producing more written text than most of human history. Probably, we are preserving a larger fraction of texts than were preserved from the Classical era. It's telling that the complaints of the loss of preservation are of the kinds of ephemera which don't exist for most periods of history. (Also telling that the person complaining about this loss of preservation is a "senior Research Software Engineer" and not a historian or archivist or similar field that actually deals with preserving or working with preserved data.)

dwheeler 4y ago
> The complete corpus of Classical Latin and Greek literature can be found in the Loeb Classical Library. Well, no. The complete surviving corpus, sure, but almost all of those works have been lost, never mind those of earlier eras. A short summary: https://en.wikipedia.org/wiki/Lost_literary_work https://en.wikipedia.org/wiki/Lost_literary_work For a specific example: I was trying to understand some of the history of logic. I learned that that in ancient Greek times Stoic logic was widely used. What was it? That is hard to answer, because practically everything is lost now. We've had to reconstruct the Stoic logic axioms from fragmentary sources, that's how bad it is. We know there was more, but we don't know exactly what it was.
- jcranmer 4y ago
  Yes, the qualifier of "surviving" is very important. Trying to look quickly for a hard number, it seems that the usual estimate (perhaps one study being widely reported rather than hard estimates) is about 5-10% of works surviving. In the modern day, I think we are doing a better of job of preservation and archival than existed in Classical times. Take something like Geocities--we had a major archival organization step in to preserve a record of what is effectively non-notable people writing about non-notable things. While I'm unsure if that particular archival record will exist in 2,000 years, I strongly suspect that there will exist some notable archive of non-notable people writing about non-notable things, the kind of record that historians today can only dream of having from those who existed 2,000 years prior to us.