3 ms·
They only allow idexing of explicitly publicly shared chats under grok.com/share/. Even if you somehow got an unpublished chat link, their robots.txt disallows
by PeterStuer 1y ago
They only allow idexing of explicitly publicly shared chats under grok.com/share/.
Even if you somehow got an unpublished chat link, their robots.txt disallows crawling it.
- optionalsquid 1y agoIf I am not mistaken (which is very possible), then until at least July 1st grok was only disallowing crawling of "/rest": https://web.archive.org/web/20250701004836/https://grok.com/robots.txt https://web.archive.org/web/20250701004836/https://grok.com/...