3 ms·
I also think it doesn't have to be that complex. I once scraped Google Search a few thousand times, in a few seconds, a few times, just for fun, and in a schoo
by aselzer 12y ago
I also think it doesn't have to be that complex.
I once scraped Google Search a few thousand times, in a few seconds, a few times, just for fun, and in a school (1000+ people).
It was quite funny when everyone suddenly saw a captcha.
Then a month ago or so I scraped Google+ for some statistics, maybe 1-3k times in 2-3 minutes and nothing happened.
The point is that my node.js scraper was far from perfect. Just a fake Chrome user agent, nothing else.
If a large organization with a static IP has a lot of users, Google is clever enough to figure out that it is large and has a high average number of requests by e.g. seeing that many people use their distinct Google Accounts.