4 ms·
I actually did notice that issue, even with using a stateful client like mechanize. Sometimes I had to scrape > 5 times in order to get through the "anti robot"
by catwind7 6y ago
I actually did notice that issue, even with using a stateful client like mechanize. Sometimes I had to scrape > 5 times in order to get through the "anti robot" page.
Other times, I get no issue at all. It's weird - maybe they're doing some pattern matching on request metadata on their end?