4 ms·
Side note: Add this to your robots.txt if you don't want ChatGPT to scrape your website: User-Agent: ChatGPT-User User-Agent: GPTBot Disallow: / Sources
by krmbzds 3y ago
Side note: Add this to your robots.txt if you don't want ChatGPT to scrape your website:
User-Agent: ChatGPT-User
User-Agent: GPTBot
Disallow: /
Sources:
- https://platform.openai.com/docs/gptbot https://platform.openai.com/docs/gptbot
- https://platform.openai.com/docs/plugins/bot https://platform.openai.com/docs/plugins/bot
- GaggiX 3y agoWell this has little to do with the article here, ChatGPT is not accessing the website itself, I even think that this feature is disable at the moment.
- deleted 3y ago[deleted]
- krmbzds 3y agoThey opened browsing feature for Plus and Enterprise users 2 days ago.
- jondwillis 3y agoSide note to your side note: the official beta ChatGPT browser plugin’s user agent is Python/3.9 aiohttp/3.8.4 (I just tested it on a domain that I control) And like a sibling comment says, any plugins/agents that are trying to scrape will have various fingerprints.