4 ms·

Glad to hear people have reached out to help, but the sad thing is for every project like this there are a hundred more that die a quiet death every year. There

Glad to hear people have reached out to help, but the sad thing is for every project like this there are a hundred more that die a quiet death every year. There's a universe of important and unique data tied to history and genealogy blogs, web databases, 20th-century software programs, and hard drive archives maintained by a single person. One of my favorites: https://fultonhistory.com/ https://fultonhistory.com/ (click on the "Go and Search My Archive" link). Tom has done an amazing job of scanning and posting old historical photos and newspaper archives, but will it last once he's no longer able to maintain it? I often hear suggestions like "let Ancestry take them over" or "the Internet Archive is the solution." I have to ask: Will either be operational in 20 years, or the data on them as easily accessible as they are now? Ancestry is currently operated by Blackstone, the third or fourth PE firm to do so in the past 15 years. Blackstone has no qualms with deleting services that no longer meet its needs (https://scottishgenes.blogspot.com/2023/08/ancestry-discontinued-its-world.html https://scottishgenes.blogspot.com/2023/08/ancestry-disconti...) or paywalling records and then jacking up the price every few years (https://www.ancestry.com/corporate/blog/were-increasing-our-monthly-subscription-prices-help-provide-you-more-content-and-new-product https://www.ancestry.com/corporate/blog/were-increasing-our-...). IA's web archive is a great resource, but it always seems to be begging for money and its founder's futile campaign against book publishers isn't helping.

Karrot_Kream 3y ago
> I often hear suggestions like "let Ancestry take them over" or "the Internet Archive is the solution." I have to ask: Will either be operational in 20 years, or the data on them as easily accessible as they are now? And on the pedestal, these words appear: My name is Ozymandias, King of Kings; Look on my Works, ye Mighty, and despair! Nothing beside remains. Round the decay Of that colossal Wreck, boundless and bare The lone and level sands stretch far away. https://poets.org/poem/ozymandias https://poets.org/poem/ozymandias
Cthulhu_ 3y ago
I think any resource of data like this - assuming it's a noncommercial project - should have an export functionality available at all times, and an ask to any visitor to download the file, archive it, and clearly label it. I like to believe that the best data repositories are the ones that are forgotten. That said, I hope someone comes up with a consumer-affordable, long-term storage medium soon.
oddmiral 3y ago
Why not to donate this data to a nearest library?
binary132 3y ago
I find myself thinking about the conundrum of long-term digital archiving pretty often. It's a very interesting question.
- Spooky23 3y ago
  You have to actively have an archive, with archivists. The geek stuff is secondary.
  - binary132 3y ago
    Something that can only survive if it has long-term funding is not really a solution to long-term archival.
- b112 3y ago
  I think the first biggie is non decaying storage. Outside of some paper, everything seems to decay fast, and even if stored correctly. Film goes bad in climate controlled vaults at times. Those DVDs made of rock supposedly last a long time, but it would be nice to have something that lasts 1000s of years.
  - binary132 3y ago
    If you're talking about analog archival that is a whole other ball of yarn, but in terms of digital then I would think that the thing to do is use only lossless compression, bit-level fidelity error correction, and multiply-redundant, multi-region storage. You must plan on storage media, providers, and entire networks partitioning, disappearing, and failing.
Fnoord 3y ago
> Glad to hear people have reached out to help, but the sad thing is for every project like this there are a hundred more that die a quiet death every year. CDROM is a terrible way to store data. Any frontend to dd could back the CDROM up 1:1. That should be the first and primary focus. After that and an upload of however long (depending on internet connection), the wisdom of the crowd could be utilized. Even if the requirement of Windows 98 is correct, any VM (perhaps even ReactOS) could access it.
- pbhjpbhj 3y ago
  IIRC, there used to be a dd-like tool, maybe cdrescue? (a Linux command-line tool), that would recheck the data extracted, and then do a "best of n" check on any spurious sectors of the disc. It's been at least 10 years since I had a CD/DVD drive though, so it might have been a mode of one of the dd_rescue/ddrescue/dd-rescue programs??
  - Fnoord 3y ago
    Probably just ddrescue by GNU, which can also be used on CD drives. For audio CDs, I've always used EAC [1] with secure mode. I believe I even used it within Wine. It yielded better results than cdrecord. Morgoth, FlashFXP, Preee, hkSFV, DCPP, and all that beautiful Windows software also worked in Wine. [1] https://www.exactaudiocopy.de/ https://www.exactaudiocopy.de/
Spooky23 3y ago
Fulton History is particularly frustrating as it’s a passion project and amazing resource, but the owner is a little… mercurial. It will absolutely die with him. Its unfortunate that when the NYS Legislature appropriated money to scan thousands (millions) of newspapers to microfiche, they did not provide any funding for digitization.
forgotmypw17 3y ago
It a lesson in how important it is to choose the most open/free and widely supported formats when creating information you want to share with the world.

Karrot_Kream 3y ago

> I often hear suggestions like "let Ancestry take them over" or "the Internet Archive is the solution." I have to ask: Will either be operational in 20 years, or the data on them as easily accessible as they are now? And on the pedestal, these words appear: My name is Ozymandias, King of Kings; Look on my Works, ye Mighty, and despair! Nothing beside remains. Round the decay Of that colossal Wreck, boundless and bare The lone and level sands stretch far away. https://poets.org/poem/ozymandias https://poets.org/poem/ozymandias

Cthulhu_ 3y ago

I think any resource of data like this - assuming it's a noncommercial project - should have an export functionality available at all times, and an ask to any visitor to download the file, archive it, and clearly label it. I like to believe that the best data repositories are the ones that are forgotten. That said, I hope someone comes up with a consumer-affordable, long-term storage medium soon.

oddmiral 3y ago

Why not to donate this data to a nearest library?

binary132 3y ago

I find myself thinking about the conundrum of long-term digital archiving pretty often. It's a very interesting question.

Spooky23 3y ago
You have to actively have an archive, with archivists. The geek stuff is secondary.
- binary132 3y ago
  Something that can only survive if it has long-term funding is not really a solution to long-term archival.
b112 3y ago
I think the first biggie is non decaying storage. Outside of some paper, everything seems to decay fast, and even if stored correctly. Film goes bad in climate controlled vaults at times. Those DVDs made of rock supposedly last a long time, but it would be nice to have something that lasts 1000s of years.
- binary132 3y ago
  If you're talking about analog archival that is a whole other ball of yarn, but in terms of digital then I would think that the thing to do is use only lossless compression, bit-level fidelity error correction, and multiply-redundant, multi-region storage. You must plan on storage media, providers, and entire networks partitioning, disappearing, and failing.

Fnoord 3y ago

> Glad to hear people have reached out to help, but the sad thing is for every project like this there are a hundred more that die a quiet death every year. CDROM is a terrible way to store data. Any frontend to dd could back the CDROM up 1:1. That should be the first and primary focus. After that and an upload of however long (depending on internet connection), the wisdom of the crowd could be utilized. Even if the requirement of Windows 98 is correct, any VM (perhaps even ReactOS) could access it.

pbhjpbhj 3y ago
IIRC, there used to be a dd-like tool, maybe cdrescue? (a Linux command-line tool), that would recheck the data extracted, and then do a "best of n" check on any spurious sectors of the disc. It's been at least 10 years since I had a CD/DVD drive though, so it might have been a mode of one of the dd_rescue/ddrescue/dd-rescue programs??
- Fnoord 3y ago
  Probably just ddrescue by GNU, which can also be used on CD drives. For audio CDs, I've always used EAC [1] with secure mode. I believe I even used it within Wine. It yielded better results than cdrecord. Morgoth, FlashFXP, Preee, hkSFV, DCPP, and all that beautiful Windows software also worked in Wine. [1] https://www.exactaudiocopy.de/ https://www.exactaudiocopy.de/

Spooky23 3y ago

Fulton History is particularly frustrating as it’s a passion project and amazing resource, but the owner is a little… mercurial. It will absolutely die with him. Its unfortunate that when the NYS Legislature appropriated money to scan thousands (millions) of newspapers to microfiche, they did not provide any funding for digitization.

forgotmypw17 3y ago

It a lesson in how important it is to choose the most open/free and widely supported formats when creating information you want to share with the world.