3 ms·Robotcop: Enforcing robots.txt and stopping bots before they reach your website2 points by jgrahamc 2y agodeleted 2y ago[deleted]