25 ms·
Ostensibly, yes. https://platform.openai.com/docs/gptbot/disallowing-gptbot https://platform.openai.com/docs/gptbot/disallowing-gptbot I would not be the slig
by arcanemachiner 3y ago
Ostensibly, yes.
https://platform.openai.com/docs/gptbot/disallowing-gptbot https://platform.openai.com/docs/gptbot/disallowing-gptbot
I would not be the slightest bit surprised if some "error" prevented this from working as intended, but hey, what do I know?
- blitzar 3y agoMaybe GPTBot's distant cousin "No user agent" will come visit just a few minutes later.
- NoZebra120vClip 3y agoYes, I'm skeptical of the headline. Are they blocking a UA or what? It was going to be the LLM-scrapers ignoring robots.txt in order to promote some kind of new standard of discouraging them. The scrapers are all rewriting their privacy policies to just say "all your text is belong to us" so I don't see much recourse for the media outlets, aside from being large with lots of aggressive IP attorneys. And what becomes of all the archive.is pages we're feeding to them anyway?