4 ms·

favicons? Try scraping the page title of a random link The consistent problem for me is that cloudflare doesnt like (some/certain/it seems random) ip ranges or

favicons? Try scraping the page title of a random link The consistent problem for me is that cloudflare doesnt like (some/certain/it seems random) ip ranges or user agents and pops up a captcha immediately

simplecto 7y ago
Have you tried proxy-rotation services like Crawlera? This is on my radar if I encounter too many issues. https://scrapinghub.com/crawlera https://scrapinghub.com/crawlera