8 ms·

archive.org feels like an irreplaceable treasure, the Wayback Machine alone is a time capsule of our digital history. I donate to them monthly and know a lot o

archive.org feels like an irreplaceable treasure, the Wayback Machine alone is a time capsule of our digital history. I donate to them monthly and know a lot of other people do as well, so I don't worry much about their financial stability. I'm more worried about external pressures taking content down. I hope the data is backed up six ways to sunday, and that somewhere there's a plan to make it all accessible if Internet Archive can't continue to play the role it does.

chrisseaton 6y ago
> archive.org feels like an irreplaceable treasure Yes I save some dead sites that I think are critically important resources to tarballs that I'm storing separately in case the Wayback Machine disappears.
- aesh2Xa1 6y ago
  That's useful for personal use. I think that the IPFS is designed for this purpose as well, but meant to be accessible to anyone.
  - chrisseaton 6y ago
    Well the idea is I can provide it publicly (not sure how... don't know how they get the copyright to do it) myself in the future if needed.
- im3w1l 6y ago
  While that may work for your personal usecase, there is a significant issue around trust, if you were to share these tarballs with anyone else. I wouldn't trust you to give me originals in the same way I trust the wayback machine. So even if there are backups lying around here and there, their utility is decreased a lot.
  - fermienrico 6y ago
    By your philosophy, just get off the internet. At some point you gotta trust HN, your keyboard driver, CPU firmware, motherboard, ISP, Google or whatever service you're using. I am not even mentioning your mailman and your amazon delivery driver. Your stance is pretty extreme - please don't discourage people from making backups.
    - bigyikes 6y ago
      I don’t think their stance is extreme as you make it seem. Insert their username into the same list you just provided: im3w1l, HN, your keyboard driver, CPU, firmware, motherboard, ISP, Google One of these things is not like the others.
dmd 6y ago
I'm very worried about its backups and what happens when the next Big One hits SF. As far as I've ever been able to determine from talking to anyone at IA (e.g., Kahle, Scott), they don't really have any sort of backups that could actually be restored from in a disaster situation.
- britmob 6y ago
  There is http://iabak.archiveteam.org http://iabak.archiveteam.org, but it’s not exactly large.
  - microcolonel 6y ago
    If I'm reading that correctly, it would only cost a bit over 500 bucks a month to host that whole archive on BackBlaze B2. Furthermore it would not be so hard to translate Archive.org items to IPFS objects, if there were an effort to pin a significant number of them to storage and network.
    - dmd 6y ago
      You're reading it correctly, but IABAK backs up 0.2% of the Internet Archive.
    - Endlessly 6y ago
      Since numbers are not 100% clear... (50 petabytes * 0.2% = 100 terabytes) [$0.005 ($/GB/Month) BackBlaze cost] [(50 petabytes) / (1 gigabyte) = 50,000,000] (50,000,000 * $0.005 = $250,000 US$) ————— Meaning based on my numbers, that is $250,000 USD a month to host 50 petabytes of data on BackBlaze.
      - ramraj07 6y ago
        That's a fraction of the AWS bills for many startups arguably doing absolutely nothing
        6y ago
erichocean 6y ago
They should team up with Backblaze...
city41 6y ago
They have lots of copyrighted content. Such as virtually every commercial retro video game ever made it seems. I personally think it’s great but surely companies aren’t too pleased? How does archive.org avoid being sued into oblivion?
- judge2020 6y ago
  It's a non-profit library and they don't publicly provide a way to download most of the copyrighted content, so rights holders aren't too concerned. I imagine most people, even copyright lawyers, personally support the free archiving of everything made in the current age as long as it doesn't detract from current business operations and the copyrights are respected for their duration.
  - city41 6y ago
    It's very easy to download the copyrighted content. For example, here is a full NES ROM set: https://archive.org/details/NESrompack https://archive.org/details/NESrompack
- beerandt 6y ago
  As far as I know, they limit public access to basically any material with a complaint or request, but keep copies. They may do more, but that seems to be the default response. Which seems smart enough: minimize litigation costs while not losing content permanently via court orders or dmcas or similar threats. But it's frustrating for the wayback machine- where sites like Snopes and some newspapers have opted out of having their history published (after being accused of ghost edits to articles). I don't know what the right answer is, but even if they didn't display the page, it would be nice to see the diffs (ala wikipedia), or at least show if and when changes to a page were made. Maybe that's beyond their mission scope, though.
  - TeMPOraL 6y ago
    > sites like Snopes and some newspapers have opted out of having their history published (after being accused of ghost edits to articles) Is there any way to interpret this behavior as anything other than straight up admitting to being dishonest? Especially in context of accusations that could be easily disproved using the Wayback Machine... unless there is something to those accusations, that is.
Aeolun 6y ago
It’s the only organization I actually donate to. Partially because it was so easy to set up a monthly donation ;)
Cthulhu_ 6y ago
They could set up a torrent that people could download parts of, I think it's an ideal system for a distributed backup. But it probably needs tweaking for that purpose; for one you need to ensure that the data is evenly distributed, and second you're dealing with data that is appended to regularly. But I think it can be done.
- Qub3d 6y ago
  It exists! The ArchiveTeam had a project, you can see its status here: http://iabak.archiveteam.org/ http://iabak.archiveteam.org/ Here's an overview: https://www.archiveteam.org/index.php?title=INTERNETARCHIVE.BAK https://www.archiveteam.org/index.php?title=INTERNETARCHIVE.... Sadly, it seems the project has fallen out of repair.
bane 6y ago
I've thought for a long time why I absolutely agree with this sentiment and the best I can come up with is that the Internet Archive feels to me like the embodiment of the old internet, not this marketing driven, data steal-and-sell, VC backed cyberpunk dystopia Internet that we have everywhere else. It's the same sort of feeling as I why I enjoy this site, wikipedia, and so on. Bonus, IA is also a bit of an organizational mess, but rewards the adventurer with rich treasure. (I was recently on a Korean history kick and came across not only 1, but several entirely different first hand books written by visitors in the late 19th and early 20th century, scanned in, freely accessible/downloadable, in a variety of formats, and with an excellent on-web reader. These books are so out of print I checked with three local counties for copies and none of them even have references to any of them in their catalog -- treasure!)
lunchables 6y ago
There are only two sites I donate to: wikipedia and archive.org. I honestly don't know what I'd do without them. I have been able to find web sites from 20 years ago on archive.org, it's an absolute treasure.

chrisseaton 6y ago

> archive.org feels like an irreplaceable treasure Yes I save some dead sites that I think are critically important resources to tarballs that I'm storing separately in case the Wayback Machine disappears.

aesh2Xa1 6y ago
That's useful for personal use. I think that the IPFS is designed for this purpose as well, but meant to be accessible to anyone.
- chrisseaton 6y ago
  Well the idea is I can provide it publicly (not sure how... don't know how they get the copyright to do it) myself in the future if needed.
im3w1l 6y ago
While that may work for your personal usecase, there is a significant issue around trust, if you were to share these tarballs with anyone else. I wouldn't trust you to give me originals in the same way I trust the wayback machine. So even if there are backups lying around here and there, their utility is decreased a lot.
- fermienrico 6y ago
  By your philosophy, just get off the internet. At some point you gotta trust HN, your keyboard driver, CPU firmware, motherboard, ISP, Google or whatever service you're using. I am not even mentioning your mailman and your amazon delivery driver. Your stance is pretty extreme - please don't discourage people from making backups.
  - bigyikes 6y ago
    I don’t think their stance is extreme as you make it seem. Insert their username into the same list you just provided: im3w1l, HN, your keyboard driver, CPU, firmware, motherboard, ISP, Google One of these things is not like the others.

dmd 6y ago

I'm very worried about its backups and what happens when the next Big One hits SF. As far as I've ever been able to determine from talking to anyone at IA (e.g., Kahle, Scott), they don't really have any sort of backups that could actually be restored from in a disaster situation.

britmob 6y ago
There is http://iabak.archiveteam.org http://iabak.archiveteam.org, but it’s not exactly large.
- microcolonel 6y ago
  If I'm reading that correctly, it would only cost a bit over 500 bucks a month to host that whole archive on BackBlaze B2. Furthermore it would not be so hard to translate Archive.org items to IPFS objects, if there were an effort to pin a significant number of them to storage and network.
  - dmd 6y ago
    You're reading it correctly, but IABAK backs up 0.2% of the Internet Archive.
  - Endlessly 6y ago
    Since numbers are not 100% clear... (50 petabytes * 0.2% = 100 terabytes) [$0.005 ($/GB/Month) BackBlaze cost] [(50 petabytes) / (1 gigabyte) = 50,000,000] (50,000,000 * $0.005 = $250,000 US$) ————— Meaning based on my numbers, that is $250,000 USD a month to host 50 petabytes of data on BackBlaze.
    - ramraj07 6y ago
      That's a fraction of the AWS bills for many startups arguably doing absolutely nothing
      - 6y ago

erichocean 6y ago

They should team up with Backblaze...

city41 6y ago

They have lots of copyrighted content. Such as virtually every commercial retro video game ever made it seems. I personally think it’s great but surely companies aren’t too pleased? How does archive.org avoid being sued into oblivion?

judge2020 6y ago
It's a non-profit library and they don't publicly provide a way to download most of the copyrighted content, so rights holders aren't too concerned. I imagine most people, even copyright lawyers, personally support the free archiving of everything made in the current age as long as it doesn't detract from current business operations and the copyrights are respected for their duration.
- city41 6y ago
  It's very easy to download the copyrighted content. For example, here is a full NES ROM set: https://archive.org/details/NESrompack https://archive.org/details/NESrompack
beerandt 6y ago
As far as I know, they limit public access to basically any material with a complaint or request, but keep copies. They may do more, but that seems to be the default response. Which seems smart enough: minimize litigation costs while not losing content permanently via court orders or dmcas or similar threats. But it's frustrating for the wayback machine- where sites like Snopes and some newspapers have opted out of having their history published (after being accused of ghost edits to articles). I don't know what the right answer is, but even if they didn't display the page, it would be nice to see the diffs (ala wikipedia), or at least show if and when changes to a page were made. Maybe that's beyond their mission scope, though.
- TeMPOraL 6y ago
  > sites like Snopes and some newspapers have opted out of having their history published (after being accused of ghost edits to articles) Is there any way to interpret this behavior as anything other than straight up admitting to being dishonest? Especially in context of accusations that could be easily disproved using the Wayback Machine... unless there is something to those accusations, that is.

Aeolun 6y ago

It’s the only organization I actually donate to. Partially because it was so easy to set up a monthly donation ;)

Cthulhu_ 6y ago

They could set up a torrent that people could download parts of, I think it's an ideal system for a distributed backup. But it probably needs tweaking for that purpose; for one you need to ensure that the data is evenly distributed, and second you're dealing with data that is appended to regularly. But I think it can be done.

Qub3d 6y ago
It exists! The ArchiveTeam had a project, you can see its status here: http://iabak.archiveteam.org/ http://iabak.archiveteam.org/ Here's an overview: https://www.archiveteam.org/index.php?title=INTERNETARCHIVE.BAK https://www.archiveteam.org/index.php?title=INTERNETARCHIVE.... Sadly, it seems the project has fallen out of repair.

bane 6y ago

I've thought for a long time why I absolutely agree with this sentiment and the best I can come up with is that the Internet Archive feels to me like the embodiment of the old internet, not this marketing driven, data steal-and-sell, VC backed cyberpunk dystopia Internet that we have everywhere else. It's the same sort of feeling as I why I enjoy this site, wikipedia, and so on. Bonus, IA is also a bit of an organizational mess, but rewards the adventurer with rich treasure. (I was recently on a Korean history kick and came across not only 1, but several entirely different first hand books written by visitors in the late 19th and early 20th century, scanned in, freely accessible/downloadable, in a variety of formats, and with an excellent on-web reader. These books are so out of print I checked with three local counties for copies and none of them even have references to any of them in their catalog -- treasure!)

lunchables 6y ago

There are only two sites I donate to: wikipedia and archive.org. I honestly don't know what I'd do without them. I have been able to find web sites from 20 years ago on archive.org, it's an absolute treasure.