3 ms·
Where did you see the big crawlers don't read it? Anthropic does.. they're pretty big.
by prismlfx 4mo ago
Where did you see the big crawlers don't read it? Anthropic does.. they're pretty big.
- graemep 4mo agoIf I look at the logs of sites that Anthropic crawls (and in some cases references in answers) and there are no requests for llms.txt