> Add a robots.txt Likely not the cause but not having a robots.txt can cause weird crawling problems. Since it takes two minutes to set up, why not. And, her

> Add a robots.txt Likely not the cause but not having a robots.txt can cause weird crawling problems. Since it takes two minutes to set up, why not. And, here's a tip, don't try and be clever, keep it simple: User-agent: \* Disallow: