Interesting. I wasn't aware that it no longer applies retroactively. Even so, medium.com's robots.txt still doesn't try to block new crawling by the Internet Ar

Interesting. I wasn't aware that it no longer applies retroactively. Even so, medium.com's robots.txt still doesn't try to block new crawling by the Internet Archiver: https://medium.com/robots.txt https://medium.com/robots.txt Or via WBM for posterity: https://web.archive.org/web/20180430183503/https://medium.com/robots.txt https://web.archive.org/web/20180430183503/https://medium.co... It seems unlikely to me that they would deliberately go to this length to prevent archival, yet not attempt to prevent it happening to begin with. Furthermore, as mentioned in your link, they still accept removal requests via email.