2 ms·Nobody actually has to comply with robots.txt.by JohnFen 3y agoNobody actually has to comply with robots.txt.