5 ms·
Anyone knows a better alternative to luminati.io? We've been using it in our company but it feels expensive. I thought this thread may be good place to ask for
by batuhanicoz 11y ago
Anyone knows a better alternative to luminati.io? We've been using it in our company but it feels expensive.
I thought this thread may be good place to ask for an alternative.
- Scoundreller 11y ago> We've been using it in our company but it feels expensive. I just looked up the cost and no kidding at $20/GB.
- tonyhb 11y agoSpin up VPS instances across multiple cities, countries and continents. Hook them up with Docker and connect them with Swarm. Label them with an IP/city/country/continent combination. Use Docker Swarm's affinity labelling to start instances in a particular city when needed. Additionally record the last IPs used and use Swarm to not deploy to those servers.
- batuhanicoz 11y agoWe scrape more than 120,000 web pages per day. Cost of spinning up VPS instances, maintaining the software needed (to automatically close/open new ones and provision them) could be higher than the 20$/GB pricing Luminati offers.
- tonyhb 11y agoI already have the tech after 3 years experience building this. I can rebuild it for you. Contact details in my profile.
- predius 11y agoI work for Scrapinghub which has a Proxy API which may work for you: http://scrapinghub.com/crawlera http://scrapinghub.com/crawlera Pricing is a lot less and we only use our own IPs.