3 ms·on spider request send robots ToS clickthrough on fail redirect.by rolph 2y agoon spider request send robots ToS clickthrough on fail redirect.cdme 2y agoI just 403 them as best I can.