5 ms·
I just stumbled on it while searching for AI crawler user agents. I'm not sure how often it's updated. It seems fairly complete from other sources I've been lo
by johneth 3y ago
I just stumbled on it while searching for AI crawler user agents.
I'm not sure how often it's updated. It seems fairly complete from other sources I've been looking at (mainly robots.txt files for well-known publishers).
- gavinhking 3y agoIt's updated daily. Thanks for posting by the way.
- johneth 3y agoThanks for making it, a great resource!