That's not going to work. $ curl https://old.reddit.com/robots.txt User-Agent: * Disallow: / Also, even if search engines are allowed, old.reddit.com pa

That's not going to work. $ curl https://old.reddit.com/robots.txt User-Agent: * Disallow: / Also, even if search engines are allowed, old.reddit.com pages are not canonical (<link rel="canonical"> points to the www.reddit.com version, which is actually reasonable behavior), so pages there would not be crawled as often or at all.