3 ms·

yeh... https://www.bing.com/search?q=https%3A%2F%2Fshoprocket.io%2Femail-confirmation%2F&qs=n&form=QBRE&sp=-1&pq=shoprocket.io%2Femail-verification&sc=1-32&sk=

• scandinavian 4y ago
  The URL in the search is this: https://shoprocket.io/email-confirmation/34b35b1 https://shoprocket.io/email-confirmation/34b35b1... I don't see that in the robots.txt https://shoprocket.io/robots.txt https://shoprocket.io/robots.txt User-agent: * Disallow: /cdn-cgi/l/email-protection Disallow: /login Disallow: /register Disallow: /404 Am I missing something?
  • rsbadger 4y ago
    You may be seeing a stale version, try this: https://shoprocket.io/robots.txt?bypass=1 https://shoprocket.io/robots.txt?bypass=1 (I made a lot of changes today when testing all, including "visit as Bingbot" from their webmaster tools with and without the URL blocked by robots.txt)