3 ms·

Robots.txt doesn't restrict anything. It's just a request of what should and shouldn't be scraped by search engine spiders.