4 ms·
I'm using Puppeteer with Chromium to mimic a browser with the headers changed to look like a real user. The proxy service only handles the data transport. Runni
by tndibona 4y ago
I'm using Puppeteer with Chromium to mimic a browser with the headers changed to look like a real user. The proxy service only handles the data transport. Running a browser cluster at scale costs me CPU, and memory (My peak yesterday on HN was 40-100 requests a second). Direct GET / POST requests to Zillow didn't work. I'm not sure $5 per month VM will cut it. Do you have the details on a provider that will give me a $5/m deal for this setup ?
- vgeek 4y agoCheck out residential proxy services. Some of them have apps users install, others may be more... nefarious. They're cheap and you can rotate through them. Some may have clients that are a pain to work with, some may be SOCKS access. https://www.webscrapingapi.com/top-residential-proxy-providers/ https://www.webscrapingapi.com/top-residential-proxy-provide... obviously SEO'd page, but it has useful info.