3 ms·
My understanding of their robots.txt policy is that it only removes the public archive of the site and they still have a private archive. That said, I'm workin
by Natsu 9y ago
My understanding of their robots.txt policy is that it only removes the public archive of the site and they still have a private archive. That said, I'm working from memory here and I wasn't able to find anything in their FAQ to confirm this, so it's possible I've misremembered something, though I did find some forum posts saying that formerly-available archives returns a 403 instead of a 404, which might provide some evidence for this.