28 ms·
Perhaps you can add a check for ai.txt as well? Not sure how it compares with just having a robots.txt, though.
by ic4ru5 3mo ago
Perhaps you can add a check for ai.txt as well? Not sure how it compares with just having a robots.txt, though.
- cwbuilds 3mo agoOkay, will do - thanks.