2 ms·They've moved from robots.txt to blocking by headers.by crazysim 4y agoThey've moved from robots.txt to blocking by headers.