2 ms·Also Googlebot is an AI training crawler so if you block AI training crawlers you should just block it.by inigyou 3mo agoAlso Googlebot is an AI training crawler so if you block AI training crawlers you should just block it.