3 ms·
It probably nudges the crawler to come rather than sending the page. If it's private, the crawler won't be able to see it.
by Kye 3y ago
It probably nudges the crawler to come rather than sending the page. If it's private, the crawler won't be able to see it.
- tivert 3y ago> It probably nudges the crawler to come rather than sending the page. If it's private, the crawler won't be able to see it. I'm sure that's exactly what it does. It is definitely not archiving some Facebook pages I've been browsing (that I would actually like to archive). It's only been capturing the login screen or an error about abusing the site.