3 ms·Same. And my logs say that AI crawls my pages roughly 1.2 bazillion times per day.by beej71 1y agoSame. And my logs say that AI crawls my pages roughly 1.2 bazillion times per day.