> If that means placating copyright holders it’s worth the cost. In many cases it's nothing to do with the copyright holder. In fact, the complete opposite: s

> If that means placating copyright holders it’s worth the cost. In many cases it's nothing to do with the copyright holder. In fact, the complete opposite: someone who subsequently bought the domain and unwittingly stuck a generic robots.txt on the site. If the IA hid content upon receiving a request from webmaster@domain then that'd make sense. But doing it automatically and retrospectively from robots.txt is bizarre and shows again the dangers of centralisation on the Internet. I now block IA from my sites as a protest. We need more competition and fresh ideas in digital archiving.