5 ms·

> First, donate to the Internet Archive: http://archive.org/donate/ http://archive.org/donate/ This is a good step, but there's one massive problem with the In

> First, donate to the Internet Archive: http://archive.org/donate/ http://archive.org/donate/ This is a good step, but there's one massive problem with the Internet Archive: If the domain registration lapses (or is sold), the new owner can direct the Internet Archive to remove archived copies of the site previously hosted on that domain[0]. In other words, even if the archives are there today, they may not be there when you want them a year (or more) from now). I've been bitten by this in the past. This means that the Internet Archive wouldn't have been able to prevent or mitigate some of the cases of link rot described here, as the content could have been scrubbed from the archive. EDIT In case it wasn't clear, I still support the Internet Archive - I just want to make sure people know this issue exists and that it's not a "silver bullet" for this problem. [0] I'm fairly certain this is still the case; if they have changed their policy recently I would be pleasantly surprised.

idlewords 12y ago
It would shock me if there's a working copy of 'rm' allowed anywhere near the Internet Archive. They take this stuff down for compliance, but my dream is that the data lives on, waiting for a saner day when the legal climate for archivists gets a little warmer.
- TazeTSchnitzel 12y ago
  Unless it's child porn, say.
- ghaff 12y ago
  It's tough though--supportive as I am of the Internet Archive's goals. How is an "archivist" different from a random individual who scrapes stuff of the Internet and rehosts it? In the aggregate, the Internet Archive looks different from the typical person who is copying articles and blog posts, wrapping them up in ads, and displaying them. The IA is non-profit, doesn't run ads, etc. It also respects robots.txt. But it's not that clear to me what the legal regime would be that allows the Internet Archive to function free and clear and doesn't hit cases that most would agree are shady.
  - DanBC 12y ago
    It's acting like a library, which have often wanted to have a copy of everything to allow people to research it. These often have exemptions written into law to allow them to do what they do, so I hope the IA is covered.
    - tedunangst 12y ago
      But libraries can't freely republish out of print books, which is about what the IA is doing. The equivalent would require the archive to have a room you'd visit with a terminal connected to the archive.
  - vitovito 12y ago
    The only difference between science and screwing around is writing it down. You can be trained as an archivist. You can get a Masters and a PhD in archival practice. There are industry-standard procedures and codes of ethics. There's a very specific understanding of what is important to save, how to save it, and how to document its context and its provenance. That's why the Internet Archive requires a certain fidelity of capture (WARCs) that a screenshot service or a citation tool don't provide. That's also why they are legally a library. Libraries have particular copyright exemptions for preservation. A typical person doesn't. But you generally have the right to make backups for your own use, and so you can also donate those backups. It's like if you were a famous person, and you bought a newspaper and a book, and when you died your personal effects were donated to your alma mater who put on a big exhibit of your life and times, that newspaper (your backup of the original that lives in the hard drives of the publisher) and that book (your backup of the original that some author wrote) are there, too. No-one's conferring any rights to the content; the publisher still owns the newspaper, and the author still owns the book, but that was your copy that is now available for everyone to see.
    - deleted 12y ago
      [deleted]
    - ghaff 12y ago
      >That's also why they are legally a library. Citation? I'm not aware of the IA having any special status. A physical newspaper or book isn't a backup. It's a physical artifact that falls under first sale doctrine. The same doesn't apply to digital.
vitovito 12y ago
Hi, author here. You're right, of course. Of the 77 210 OK But Gone URLs, 35 had copies in the archive (more had copies than that, but only 35 were from the right year), but 5 (three domains total) had been blocked by a later robots.txt, and 1 had been "excluded" from the Wayback Machine, which I assume is the removal option you mention. That's 6 redacted URLs of 77, versus 35 good ones. I'm not going to not donate to the Internet Archive and help preserve access to those 35 because of those 6. EDIT: In the interest of rigor, of the 76 404 Not Found URLs, 4 URLs (three domains) had been blocked by a later robots.txt. 45 had relevant content preserved and accessible. That's 10 redactions total. Also, when the Internet Archive imports third-party captures, like those from Archive Team, they are included irrespective of the robots.txt from the time of capture, which is then only used to manage display of the content. It's LOCKSS, lots of copies keeps stuff safe. The Internet Archive isn't the only solution; we need more archives. But it's a start, and getting all the bookmarking services to contribute to it, and to improve and make public their caches would be a great improvement.
polynomial 12y ago
I'm not sure I understand this. Just because I buy/sell a domain in 2014, that doesn't give me the right to retroactively change what was there in 2007. Would like to hear more about the basis for such a policy, whose only use case seems like censorship of the past, which Archive is clearly not aligned with.
- deleted 12y ago
  [deleted]
- maxerickson 12y ago
  Speculating on motivations, they are resource constrained. Spending resources keeping thousands of pages available to the public is a reasonable tradeoff vs entering/inviting a protracted legal battle. As far as the mechanics of it, they follow a policy that explicitly recommends respecting robots.txt: https://archive.org/about/faqs.php#Rights https://archive.org/about/faqs.php#Rights http://www2.sims.berkeley.edu/research/conferences/aps/removal-policy.html http://www2.sims.berkeley.edu/research/conferences/aps/remov...
deleted 12y ago
[deleted]

idlewords 12y ago

It would shock me if there's a working copy of 'rm' allowed anywhere near the Internet Archive. They take this stuff down for compliance, but my dream is that the data lives on, waiting for a saner day when the legal climate for archivists gets a little warmer.

TazeTSchnitzel 12y ago
Unless it's child porn, say.
ghaff 12y ago
It's tough though--supportive as I am of the Internet Archive's goals. How is an "archivist" different from a random individual who scrapes stuff of the Internet and rehosts it? In the aggregate, the Internet Archive looks different from the typical person who is copying articles and blog posts, wrapping them up in ads, and displaying them. The IA is non-profit, doesn't run ads, etc. It also respects robots.txt. But it's not that clear to me what the legal regime would be that allows the Internet Archive to function free and clear and doesn't hit cases that most would agree are shady.
- DanBC 12y ago
  It's acting like a library, which have often wanted to have a copy of everything to allow people to research it. These often have exemptions written into law to allow them to do what they do, so I hope the IA is covered.
  - tedunangst 12y ago
    But libraries can't freely republish out of print books, which is about what the IA is doing. The equivalent would require the archive to have a room you'd visit with a terminal connected to the archive.
- vitovito 12y ago
  The only difference between science and screwing around is writing it down. You can be trained as an archivist. You can get a Masters and a PhD in archival practice. There are industry-standard procedures and codes of ethics. There's a very specific understanding of what is important to save, how to save it, and how to document its context and its provenance. That's why the Internet Archive requires a certain fidelity of capture (WARCs) that a screenshot service or a citation tool don't provide. That's also why they are legally a library. Libraries have particular copyright exemptions for preservation. A typical person doesn't. But you generally have the right to make backups for your own use, and so you can also donate those backups. It's like if you were a famous person, and you bought a newspaper and a book, and when you died your personal effects were donated to your alma mater who put on a big exhibit of your life and times, that newspaper (your backup of the original that lives in the hard drives of the publisher) and that book (your backup of the original that some author wrote) are there, too. No-one's conferring any rights to the content; the publisher still owns the newspaper, and the author still owns the book, but that was your copy that is now available for everyone to see.
  - deleted 12y ago
    [deleted]
  - ghaff 12y ago
    >That's also why they are legally a library. Citation? I'm not aware of the IA having any special status. A physical newspaper or book isn't a backup. It's a physical artifact that falls under first sale doctrine. The same doesn't apply to digital.

vitovito 12y ago

Hi, author here. You're right, of course. Of the 77 210 OK But Gone URLs, 35 had copies in the archive (more had copies than that, but only 35 were from the right year), but 5 (three domains total) had been blocked by a later robots.txt, and 1 had been "excluded" from the Wayback Machine, which I assume is the removal option you mention. That's 6 redacted URLs of 77, versus 35 good ones. I'm not going to not donate to the Internet Archive and help preserve access to those 35 because of those 6. EDIT: In the interest of rigor, of the 76 404 Not Found URLs, 4 URLs (three domains) had been blocked by a later robots.txt. 45 had relevant content preserved and accessible. That's 10 redactions total. Also, when the Internet Archive imports third-party captures, like those from Archive Team, they are included irrespective of the robots.txt from the time of capture, which is then only used to manage display of the content. It's LOCKSS, lots of copies keeps stuff safe. The Internet Archive isn't the only solution; we need more archives. But it's a start, and getting all the bookmarking services to contribute to it, and to improve and make public their caches would be a great improvement.

polynomial 12y ago

I'm not sure I understand this. Just because I buy/sell a domain in 2014, that doesn't give me the right to retroactively change what was there in 2007. Would like to hear more about the basis for such a policy, whose only use case seems like censorship of the past, which Archive is clearly not aligned with.

deleted 12y ago
[deleted]
maxerickson 12y ago
Speculating on motivations, they are resource constrained. Spending resources keeping thousands of pages available to the public is a reasonable tradeoff vs entering/inviting a protracted legal battle. As far as the mechanics of it, they follow a policy that explicitly recommends respecting robots.txt: https://archive.org/about/faqs.php#Rights https://archive.org/about/faqs.php#Rights http://www2.sims.berkeley.edu/research/conferences/aps/removal-policy.html http://www2.sims.berkeley.edu/research/conferences/aps/remov...

deleted 12y ago

[deleted]