4 ms·C'mon! Mistakes happen, sure. But a correctly written robots.txt would prevent thisby namank 15y agoC'mon! Mistakes happen, sure. But a correctly written robots.txt would prevent this