3 ms·
Needs more details, a couple of examples that instantly spring to mind: Do you spread requests out across multiple IP's to avoid bans / rate limiting? Do you
by Jake232 12y ago
Needs more details, a couple of examples that instantly spring to mind:
Do you spread requests out across multiple IP's to avoid bans / rate limiting?
Do you run javascript on the pages?
- notastartup 12y agoNo, it will not fly through thousands of pages at a blinding rate using multiple ip addresses, it goes about at a human like pace to avoid getting banned. If an ip is banned a new one may be assigned. If you need to go faster you will need concurrent jobs. All the jobs use chrome browser, and supports any website.