3 ms·
This information is outdated, and ia_archiver now disobeys robots.txt (see https://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-wel
by a3_nm 8y ago
This information is outdated, and ia_archiver now disobeys robots.txt (see https://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives/ https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...). You can still get your website removed from Internet Archive but you have to contact them.