3 ms·
They keep the data and just don’t display it. The last thing they need is a court order demanding they delete it. Sure the archive is useful today, but it’s pr
by slededit 8y ago
They keep the data and just don’t display it. The last thing they need is a court order demanding they delete it.
Sure the archive is useful today, but it’s primary purpose is retaining information for future generations. If that means placating copyright holders it’s worth the cost.
- rovr138 8y ago> They keep the data and just don’t display it. I’ve read about the robots.txt mentioned before but hadn’t seen this mentioned. Any idea if they have this somewhere on the site?
- slededit 8y agoSure see here: https://archive.org/post/133690/robotstxt-only-gives-temporary-removal-of-pages https://archive.org/post/133690/robotstxt-only-gives-tempora...
- dingaling 8y ago> If that means placating copyright holders it’s worth the cost. In many cases it's nothing to do with the copyright holder. In fact, the complete opposite: someone who subsequently bought the domain and unwittingly stuck a generic robots.txt on the site. If the IA hid content upon receiving a request from webmaster@domain then that'd make sense. But doing it automatically and retrospectively from robots.txt is bizarre and shows again the dangers of centralisation on the Internet. I now block IA from my sites as a protest. We need more competition and fresh ideas in digital archiving.