4 ms·
>Every URL being sent through GMAIL is being crawled by Googlebot This is demonstrably false. Email a gmail address with a URL that only you know about. Then
by rfergie 14y ago
>Every URL being sent through GMAIL is being crawled by Googlebot
This is demonstrably false.
Email a gmail address with a URL that only you know about. Then observe your server logs - you will not see GoogleBot at this URL
- fear91 14y agoI've done this before and I know this works. Tested on many thousands of URLs.
- Natsu 14y agoI thought this was pretty interesting so I just tested this by setting up a Perl script that pretends to be a web server and having it printing out the IPs of anyone who accesses it while feeding them a 403 page. After emailing myself a link to that server on Gmail, referencing a document that doesn't exist, I waited for over six hours. No hits were recorded from any source and I am unable to duplicate your results.