4 ms·

I'm so glad the download-entire-wikipedia function continues to exist. That will help counter the "lost the entire library problem" from the city of Alexandria.

I'm so glad the download-entire-wikipedia function continues to exist. That will help counter the "lost the entire library problem" from the city of Alexandria. To be fair, Wikipedia only has summaries, not the detailed material, but it's still important.

porphyra 6y ago
It is pretty awesome that there are people like /r/datahoarder that are obsessed with backing up the collective knowledge of humanity.
- capableweb 6y ago
  I'm not familiar with r/datahoarder, but if the name bears any significance, it seems they are mostly centered on hoarding data, which means just digital I guess? If so, I much rather would want to promote efforts like Internet Archive that back up all kind of things, not just digital data.
  - deleted 6y ago
    [deleted]
  - KMnO4 6y ago
    What does the Internet Archive back up that isn’t represented by 1s and 0s?
    - cguess 6y ago
      A ton of 35mm and 16mm film reels, vinyl, and even wax recording, physical books and a lot more. They make digital copies of them, but they also archive the physical versions as well. Here's a selection of the movies: https://archive.org/details/moviesandfilms?tab=about https://archive.org/details/moviesandfilms?tab=about
    - LeoPanthera 6y ago
      The Physical Archive. https://en.wikipedia.org/wiki/Internet_Archive#Physical_media https://en.wikipedia.org/wiki/Internet_Archive#Physical_medi...
  - CompuHacker 6y ago
    Promote both, rest easier.
- aarchi 6y ago
  There's also Archive Team, focused on preserving at-risk sites before being taken offline. https://wiki.archiveteam.org/ https://wiki.archiveteam.org/
tablespoon 6y ago
> I'm so glad the download-entire-wikipedia function continues to exist. That will help counter the "lost the entire library problem" from the city of Alexandria. To be fair, Wikipedia only has summaries, not the detailed material, but it's still important. Personally, I think Wikipedia's quality is too poor for that. Plus, it's digital, so when our civilization is at risk of "[losing] the entire library" it probably would have already lost the ability to maintain the computer systems to access Wikipedia dumps.
- smoldesu 6y ago
  The content on Wikipedia is really not that bad. Obviously a Wikipedia article will never be the final say on any specific subject, but it tends to do a pretty good of aggregating sources and condensing it into a reader-friendly synopsis. This data is super valuable, if not just for the sources alone.
- buzzerbetrayed 6y ago
  > so when our civilization is at risk of "[losing] the entire library" it probably would have already lost the ability to maintain the computer systems to access Wikipedia dumps But as long as it continues to exist, some future civilization could figure out how to read the data again, eventually. Just like we eventually discovered how to read ancient languages that were once forgotten.
  - tablespoon 6y ago
    > But as long as it continues to exist, some future civilization could figure out how to read the data again, eventually. Just like we eventually discovered how to read ancient languages that were once forgotten. Eh, I think you're vastly underestimating how difficult that would be. 1. The media would have to last hundreds of years at least, when it's hoped modern archival media can last maybe fifty. 2. Even assuming the media did last, the new civilization would have to reverse engineer encoding on top of encoding on top of encoding (e.g. physical disk encoding, complex filesystems, file formats, character encodings). Our civilization already has trouble reading some old file formats, and our disks already have trouble reading their data (which is why they pack a ton of error correction information). It took the Rosetta stone to figure out how to read encoding of Egyptian hieroglyphics, when that language was still alive in the form of Coptic. 3. Then you're dealing with the probability that the hard disks the future archeologists find will even have a Wikipedia dump on them. That probability will be very small, given very few people will download these dumps.
LeoPanthera 6y ago
Kiwix offers downloadable material (including the full Wikipedia), in a format specifically designed for offline browsing. https://wiki.kiwix.org/wiki/Content https://wiki.kiwix.org/wiki/Content Their Wikipedia bundle wasn't being updated for a while and had fallen out of date, but that seems to have been fixed now.
- easton 6y ago
  This is what makes the 256GB iPad worth it, you can carry the entire Wikipedia (with pictures) around with you. It’s fantastic when on a plane or if you need to entertain yourself without Wi-Fi.
  - m-p-3 6y ago
    I "upgraded" the internal storage of my Kobo Glo HD (which is in fact using a 4GB MicroSD card as its onboard storage) to a 128GB. I can store the "all maxi" 82GB ZIM file on it. The Kobo firmware includes (but doesn't advertise) ZIM support, so I can browse Wikipedia on an eInk display and great battery life without Internet connectivity. That would be a great post-apocalyptic knowledge archive, and it would be easy enough to recharge using a USB solar panel.
    - ObsoleteNerd 6y ago
      I've been wanting something like this for ages. I always thought the Kindle not having the full Wikipedia archive as an optional install was a massive missed opportunity, especially back when the Kindle had the keyboard at the bottom. It would have literally been the Hitchhikers Guide as explained in the books.
    - lxgr 6y ago
      > The Kobo firmware includes (but doesn't advertise) ZIM support This is one of the nicest hidden features/easter eggs I've ever come across. Thanks for sharing this!
    - ianbooker 6y ago
      Time to update XKCD: https://xkcd.com/548/ https://xkcd.com/548/
qwertywert_ 6y ago
Can it also download sources if available too, would be cool.
1vuio0pswjnm7 6y ago
This is a also user privacy solution that could be applied in other areas. If one is only querying a local store of Wikipedia pages, then third party observation of those queries is perhaps impossible, and so the "surveillance capital" "business model" may become infeasible. (Now this is perhaps a poor example because Wikipedia generally does not appear to be engaged in "surveillance capitalism". Maybe their straightforward provision of bulk data coupled with this fact is not a coincidence.) But there are other databases besides Wikipedia that internet users query weekly, daily, hourly or even more frequently. Making incessant queries over the internet can a rich source of data for a third party observer, one that can form the foundation of "surveillance capitalism" revenue strategies. Whereas if users can download bulk data, and thereby avoid queries over the internet, in many cases they can avoid falling victim to "surveillance capitalism". Bulk data can be a viable privacy solution.
monadic8 6y ago
There's not much evidence that the loss of the Library of Alexandria constituted much loss to human knowledge. This idea seems to come from Carl Sagan's Cosmos. While there a lot of good things to say about him he's not much of a historian.
- mixmastamyk 6y ago
  Maybe that facts such as the rough circumference of the earth were lost for a thousand years had something to do with it? Or is that false?
- orblivion 6y ago
  I'm going to ask my parents about this to see if they learned this idea in school. They didn't grow up in an English speaking country, and I don't think they're big fans of Sagan.
- dwheeler 6y ago
  I think it is almost certain that a vast amount of human knowledge was lost. The policy of the city of Alexandria was to copy every book that went through it, and give the copy back. As a result, they ended up with a spectacularly large collection of original materials. The amount of ancient literature that is no longer available is vast. History, literature, you name it, so much has been lost. The odds are very good that at least a significant portion of that were in the Alexandrian library.

porphyra 6y ago

It is pretty awesome that there are people like /r/datahoarder that are obsessed with backing up the collective knowledge of humanity.

capableweb 6y ago
I'm not familiar with r/datahoarder, but if the name bears any significance, it seems they are mostly centered on hoarding data, which means just digital I guess? If so, I much rather would want to promote efforts like Internet Archive that back up all kind of things, not just digital data.
- deleted 6y ago
  [deleted]
- KMnO4 6y ago
  What does the Internet Archive back up that isn’t represented by 1s and 0s?
  - cguess 6y ago
    A ton of 35mm and 16mm film reels, vinyl, and even wax recording, physical books and a lot more. They make digital copies of them, but they also archive the physical versions as well. Here's a selection of the movies: https://archive.org/details/moviesandfilms?tab=about https://archive.org/details/moviesandfilms?tab=about
  - LeoPanthera 6y ago
    The Physical Archive. https://en.wikipedia.org/wiki/Internet_Archive#Physical_media https://en.wikipedia.org/wiki/Internet_Archive#Physical_medi...
- CompuHacker 6y ago
  Promote both, rest easier.
aarchi 6y ago
There's also Archive Team, focused on preserving at-risk sites before being taken offline. https://wiki.archiveteam.org/ https://wiki.archiveteam.org/

tablespoon 6y ago

> I'm so glad the download-entire-wikipedia function continues to exist. That will help counter the "lost the entire library problem" from the city of Alexandria. To be fair, Wikipedia only has summaries, not the detailed material, but it's still important. Personally, I think Wikipedia's quality is too poor for that. Plus, it's digital, so when our civilization is at risk of "[losing] the entire library" it probably would have already lost the ability to maintain the computer systems to access Wikipedia dumps.

smoldesu 6y ago
The content on Wikipedia is really not that bad. Obviously a Wikipedia article will never be the final say on any specific subject, but it tends to do a pretty good of aggregating sources and condensing it into a reader-friendly synopsis. This data is super valuable, if not just for the sources alone.
buzzerbetrayed 6y ago
> so when our civilization is at risk of "[losing] the entire library" it probably would have already lost the ability to maintain the computer systems to access Wikipedia dumps But as long as it continues to exist, some future civilization could figure out how to read the data again, eventually. Just like we eventually discovered how to read ancient languages that were once forgotten.
- tablespoon 6y ago
  > But as long as it continues to exist, some future civilization could figure out how to read the data again, eventually. Just like we eventually discovered how to read ancient languages that were once forgotten. Eh, I think you're vastly underestimating how difficult that would be. 1. The media would have to last hundreds of years at least, when it's hoped modern archival media can last maybe fifty. 2. Even assuming the media did last, the new civilization would have to reverse engineer encoding on top of encoding on top of encoding (e.g. physical disk encoding, complex filesystems, file formats, character encodings). Our civilization already has trouble reading some old file formats, and our disks already have trouble reading their data (which is why they pack a ton of error correction information). It took the Rosetta stone to figure out how to read encoding of Egyptian hieroglyphics, when that language was still alive in the form of Coptic. 3. Then you're dealing with the probability that the hard disks the future archeologists find will even have a Wikipedia dump on them. That probability will be very small, given very few people will download these dumps.

LeoPanthera 6y ago

Kiwix offers downloadable material (including the full Wikipedia), in a format specifically designed for offline browsing. https://wiki.kiwix.org/wiki/Content https://wiki.kiwix.org/wiki/Content Their Wikipedia bundle wasn't being updated for a while and had fallen out of date, but that seems to have been fixed now.

easton 6y ago
This is what makes the 256GB iPad worth it, you can carry the entire Wikipedia (with pictures) around with you. It’s fantastic when on a plane or if you need to entertain yourself without Wi-Fi.
- m-p-3 6y ago
  I "upgraded" the internal storage of my Kobo Glo HD (which is in fact using a 4GB MicroSD card as its onboard storage) to a 128GB. I can store the "all maxi" 82GB ZIM file on it. The Kobo firmware includes (but doesn't advertise) ZIM support, so I can browse Wikipedia on an eInk display and great battery life without Internet connectivity. That would be a great post-apocalyptic knowledge archive, and it would be easy enough to recharge using a USB solar panel.
  - ObsoleteNerd 6y ago
    I've been wanting something like this for ages. I always thought the Kindle not having the full Wikipedia archive as an optional install was a massive missed opportunity, especially back when the Kindle had the keyboard at the bottom. It would have literally been the Hitchhikers Guide as explained in the books.
  - lxgr 6y ago
    > The Kobo firmware includes (but doesn't advertise) ZIM support This is one of the nicest hidden features/easter eggs I've ever come across. Thanks for sharing this!
  - ianbooker 6y ago
    Time to update XKCD: https://xkcd.com/548/ https://xkcd.com/548/

qwertywert_ 6y ago

Can it also download sources if available too, would be cool.

1vuio0pswjnm7 6y ago

This is a also user privacy solution that could be applied in other areas. If one is only querying a local store of Wikipedia pages, then third party observation of those queries is perhaps impossible, and so the "surveillance capital" "business model" may become infeasible. (Now this is perhaps a poor example because Wikipedia generally does not appear to be engaged in "surveillance capitalism". Maybe their straightforward provision of bulk data coupled with this fact is not a coincidence.) But there are other databases besides Wikipedia that internet users query weekly, daily, hourly or even more frequently. Making incessant queries over the internet can a rich source of data for a third party observer, one that can form the foundation of "surveillance capitalism" revenue strategies. Whereas if users can download bulk data, and thereby avoid queries over the internet, in many cases they can avoid falling victim to "surveillance capitalism". Bulk data can be a viable privacy solution.

monadic8 6y ago

There's not much evidence that the loss of the Library of Alexandria constituted much loss to human knowledge. This idea seems to come from Carl Sagan's Cosmos. While there a lot of good things to say about him he's not much of a historian.

mixmastamyk 6y ago
Maybe that facts such as the rough circumference of the earth were lost for a thousand years had something to do with it? Or is that false?
orblivion 6y ago
I'm going to ask my parents about this to see if they learned this idea in school. They didn't grow up in an English speaking country, and I don't think they're big fans of Sagan.
dwheeler 6y ago
I think it is almost certain that a vast amount of human knowledge was lost. The policy of the city of Alexandria was to copy every book that went through it, and give the copy back. As a result, they ended up with a spectacularly large collection of original materials. The amount of ancient literature that is no longer available is vast. History, literature, you name it, so much has been lost. The odds are very good that at least a significant portion of that were in the Alexandrian library.