3 ms·
Can you share that ChatGPT transcript? One guess: ChatGPT uses Bing for search queries, and your robots.txt doesn't block Bing. If that's what is happening her
by simonw 1y ago
Can you share that ChatGPT transcript?
One guess: ChatGPT uses Bing for search queries, and your robots.txt doesn't block Bing. If that's what is happening here I agree that this is really confusing and should be clarified by the OpenAI bots page.
- spacebuffer 1y agoHere you go: https://chatgpt.com/share/68bc125a-9e9c-8005-9c9f-298dbd541d3f https://chatgpt.com/share/68bc125a-9e9c-8005-9c9f-298dbd541d...
- simonw 1y agoYeah, wow that's a lot of information for a site that's supposedly blocked using robots.txt! My best guess is that this is a Bing thing - ChatGPT uses Bing as their search partner (though they don't make that very obvious at all), and BingBot isnt't blocked by your site. I think OpenAI need to be a whole lot more transparent about this. It's very misleading to block their crawlers and have it not make any difference at all to the search results returns within ChatGPT.