3 ms·

They exclude Snopes and I think Salon from archiving.

• Hitton 6y ago
  Do you have any source on that? Sites can request archive.org to stop archiving them and to delete what is currently archived. They can do it for any reason; concealing changes of article contents might be one of them.
  • dependenttypes 6y ago
    https://web.archive.org/web/*/snopes.com https://web.archive.org/web/*/snopes.com > Sorry. > This URL has been excluded from the Wayback Machine. They also do not exclude the archive.org bot in https://www.snopes.com/robots.txt https://www.snopes.com/robots.txt
    • Hitton 6y ago
      That only shows that it's excluded, not for what reason. In 2017 Internet Archive announced it will start to ignore robots.txt in the future. When I tried to archive random facebook page (it was not allowed in robots.txt), it archived it happily. Afaik current way to exclude you site requires contacting info@archive.org and proving that the site is your.