3 ms·
Yeah, people like to try and crawl Google. So they're keeping robots out. But it's actually really nice to be able to pull a result or two from ddg by crawling
by striking 9y ago
Yeah, people like to try and crawl Google. So they're keeping robots out.
But it's actually really nice to be able to pull a result or two from ddg by crawling, or to be able to use a VPN without having to solve a zillion captcha.
- Steeeve 9y agoI run into those captcha's occasionally and generally reconnecting to a different VPN makes them go away. I always assume they are tied to bot activity coming through the same IP or range. I've scraped google in the past (years and years ago) and from what I recall you end up hitting the captchas after a set number of results in too short a period. Back then it was a pretty standard SEO activity, but it's usefulness was minimized when google started localizing results.