2 ms·
There are such paid proxy services that provide you with a refreshable IP pool. Unfortunately, you can't control which IP they give you at each request (or when
by nurettin 7y ago
There are such paid proxy services that provide you with a refreshable IP pool. Unfortunately, you can't control which IP they give you at each request (or when you can, your request rate drops significantly) so they do not work with websites which require you to keep a session alive.
- bshipp 7y agoI use crawlera, and it does have the capability to establish and maintain a session. I can't pick the precise initial IP but once the connection is made I can certainly tell it to keep using that IP. Sessions are the only way to use crawlera with libraries like cloudflare-scrape, which pin your authentication to a specific IP.
- nurettin 7y agoThat is interesting. $1000 tier is a considerable amount, but perhaps one can find a profitable line of products to justify the price. Of course, this is too expensive for initial development most of the time, so I'd go for an in-house network of machines initially.