3 ms·robots.txt generally only addresses permission to crawl; not permission to reproduce the content.by wildrhythms 3y agorobots.txt generally only addresses permission to crawl; not permission to reproduce the content.