3 ms·Sites would just block the Internet Archive crawler as well.by drdexebtjl 11d agoSites would just block the Internet Archive crawler as well.