12 ms·
Good performance matters if you have decent networking infrastructure or your server has limited resources. Bandwidth and IP limits are the most common bottle
by tampo9 9y ago
Good performance matters if you have decent networking infrastructure or your server has limited resources.
Bandwidth and IP limits are the most common bottle necks, but these can be solved using multiple proxies and ssh tunnels. Colly has built in support for switching proxies [1].
[1] http://go-colly.org/docs/best_practices/distributed/ http://go-colly.org/docs/best_practices/distributed/
- mlevental 9y agothe limitation is usually rate throttling on the page >but these can be solved using multiple proxies and ssh tunnels. Colly has built in support for switching proxies interesting >your server has limited resources. possibly.
- deleted 9y ago[deleted]