3 ms·Its mostly for search engines to figure out how to crawl your website. Use it sparingly.by giancarlostoro 1y agoIts mostly for search engines to figure out how to crawl your website. Use it sparingly.