3 ms·Amateurs. If you are going to crawl distribute it across unique IP addresses.by detroitcoder 12y agoAmateurs. If you are going to crawl distribute it across unique IP addresses.