4 ms·
Spin up VPS instances across multiple cities, countries and continents. Hook them up with Docker and connect them with Swarm. Label them with an IP/city/count
by tonyhb 11y ago
Spin up VPS instances across multiple cities, countries and continents.
Hook them up with Docker and connect them with Swarm.
Label them with an IP/city/country/continent combination.
Use Docker Swarm's affinity labelling to start instances in a particular city when needed. Additionally record the last IPs used and use Swarm to not deploy to those servers.
- batuhanicoz 11y agoWe scrape more than 120,000 web pages per day. Cost of spinning up VPS instances, maintaining the software needed (to automatically close/open new ones and provision them) could be higher than the 20$/GB pricing Luminati offers.
- tonyhb 11y agoI already have the tech after 3 years experience building this. I can rebuild it for you. Contact details in my profile.