That's a really good point. I'm not a fan of Internet Archive ignoring robots.txt, but if I'm really unhappy about it I can block their robot in Apache using .

That's a really good point. I'm not a fan of Internet Archive ignoring robots.txt, but if I'm really unhappy about it I can block their robot in Apache using .htaccess rules (as long as they continue using archive.org_bot as their user agent).