4 ms·Robots.txt isnt a blocklist; more of polite suggestions you have to hope are respected.by deadly_syn 3y agoRobots.txt isnt a blocklist; more of polite suggestions you have to hope are respected.