3 ms·
The URL in the search is this: https://shoprocket.io/email-confirmation/34b35b1 https://shoprocket.io/email-confirmation/34b35b1... I don't see that in the rob
by scandinavian 4y ago
The URL in the search is this: https://shoprocket.io/email-confirmation/34b35b1 https://shoprocket.io/email-confirmation/34b35b1...
I don't see that in the robots.txt https://shoprocket.io/robots.txt https://shoprocket.io/robots.txt
User-agent: *
Disallow: /cdn-cgi/l/email-protection
Disallow: /login
Disallow: /register
Disallow: /404
Am I missing something?
- rsbadger 4y agoYou may be seeing a stale version, try this: https://shoprocket.io/robots.txt?bypass=1 https://shoprocket.io/robots.txt?bypass=1 (I made a lot of changes today when testing all, including "visit as Bingbot" from their webmaster tools with and without the URL blocked by robots.txt)