3 ms·
Now there's a nice bit of irony .. archive.org doesn't respect robots.txt, is nearly impossible to block, but has an anti-bot wall?
by elric 4y ago
Now there's a nice bit of irony .. archive.org doesn't respect robots.txt, is nearly impossible to block, but has an anti-bot wall?
- misssocrates 4y agoIf know the right people on the inside they'll scrub the archive for you. https://web.archive.org/web/20230000000000*/https://twitter.com/TaylorLorenz https://web.archive.org/web/20230000000000*/https://twitter....
- bombcar 4y agoThat is when you have to go to the Deep Archives™ (completely distinct from the Depp Archives®).
- mtrycz2 4y agoThe anti-robot feature to me looks like Bloomberg's, not Archive's