3 ms·

Can someone explain what exactly he meant by "throttling the google bot"?

• newman314 15y ago
  It's telling googlebot to slow down and not be so aggressive. You can use google webmaster tools to dial down the crawl rate. Some think you can use Crawl-delay: in robots.txt but I have not tested this (and it may very well not work).