3 ms·
This is good work. I thought there was no way to do this since Google Search blocks bots.
by charred_toast 9y ago
This is good work. I thought there was no way to do this since Google Search blocks bots.
- slackingoff2017 9y agoThey let some obvious bots through if you go slow enough. A lot of SEO and AdWords tracking companies still do unofficially tolerated automated google search. I assume allowing some low amount of obvious botting prevents people from developing really sneaky bots that are much harder to block. It's probably to prevent an arms race that google may not be able to win.
- i336_ 9y agoWhat an incredibly good point. That gives me hope, as someone who wants to do low-frequency data analysis that's only possible via multiple queries.
- charred_toast 9y agoI redundantly made the following comment upon referring to this comment later in this thread: Try Google Custom Search as well, $100 a year for 20k queries; I'm a client.
- slackingoff2017 9y agoGood evening, I regret to inform you that Google custom search is dead. http://fortune.com/2017/02/21/google-site-search-discontinued/ http://fortune.com/2017/02/21/google-site-search-discontinue... The good news it's that it's reborn as an ad support version. Google doesn't want your dirty cash when your competitors are willing to pay much more for ad placement on your site.
- primozk 9y agoGoogle Site Search is dead, but Google Custom Search is not.
- slackingoff2017 9y agoYou might be okay, but I would try to get some insider info on how AdWords Analytics companies get away with it. Might just be sneaky bots. A lot of idiots write code that pings or loads google.com to check for internet access so they can't block it completely.