27 ms·
llms.txt files have nothing to do with crawlers or big LLM companies. They are for individual client agents to use. I have my clients set up to always use them
by jph00 8mo ago
llms.txt files have nothing to do with crawlers or big LLM companies. They are for individual client agents to use. I have my clients set up to always use them when they’re available, and since I did that they’ve been way faster and more token efficient when using sites that have llms.txt files.
So I can absolutely assure you that LLM clients are reading them, because I use that myself every day.
- reconnecting 8mo agoThanks for the clarification. >for use in LLMs such as Claude (1) From your website, it seems to me that LLMs.txt is addressed to all LLMs such as Claude, not just 'individual client agents' . Claude never touched LLMs.txt on my servers, hence the confusion. 1. https://llmstxt.org https://llmstxt.org