2 ms·

I would add not just rely on robots.txt - use no index and/or password protect. Robots stops crawling not indexing