2 ms·
It's not wiped (they don't delete anything) but if robots.txt forbids indexing then access to old archives is blocked. It can be quite aggravating, but at least
by lorenzhs 10y ago
It's not wiped (they don't delete anything) but if robots.txt forbids indexing then access to old archives is blocked. It can be quite aggravating, but at least the information is not lost---if robots.txt or archive.org's policy changes, it can be retrieved again.