4 ms·
That's the smartest thing I saw in quite a while
by mapcars 4mo ago
That's the smartest thing I saw in quite a while
- graemep 4mo agoDoes it work though? The big LLM crawlers do not read llms.txt so will they read and follow the same instructions as HTML?
- prismlfx 4mo agoWhere did you see the big crawlers don't read it? Anthropic does.. they're pretty big.
- graemep 4mo agoIf I look at the logs of sites that Anthropic crawls (and in some cases references in answers) and there are no requests for llms.txt
- mapcars 4mo agoI have no idea, in theory it might catch some miss-configured agents off-guard
- Aboutplants 4mo agoSomeone has to have done or is doing an experiment with this right? I also think that if it was an actual profitable thing then we would know about it pretty quickly. It would pop up everywhere
- iamacyborg 4mo agoApparently new checks in Chrome Lighthouse are checking for the existence of the file. https://searchengineland.com/google-llms-txt-chrome-lighthouse-478246 https://searchengineland.com/google-llms-txt-chrome-lighthou...
- patwards 4mo agoYeah I want to know how many donations they get
- flexagoon 4mo agoThis is not for LLM data crawlers, it's for LLM agents the end users are running on their machines