2 ms·
I wonder if Google indexes with browser user agents sometimes: unless the site used source IP tricks, it’d let Google detect this sort of thing.
by fiddlerwoaroof 5y ago
I wonder if Google indexes with browser user agents sometimes: unless the site used source IP tricks, it’d let Google detect this sort of thing.
- IX-103 5y agoSupposedly the Chrome safe browsing service crawls the web looking for malware. Since that's focused on finding malware they likely make it look like a normal user in terms of user agent and IP address, so it can probably detect these pages. I believe the signals from the safe browsing service are used to affect search result ranking, but I have no idea if they look for different content being served to the Google crawler.