4 ms·They really missed an opportunity to get creative with their own `robots.txt` implementation.by ajcp 6y agoThey really missed an opportunity to get creative with their own `robots.txt` implementation.