2 ms·

This, to me, is the strongest argument to offer these slop generators. It provides an incentive to follow the robots.txt.

• xyzal 6mo ago
  Exactly. You disobey robots file => we'll make your crawl gain a net negative.