3 ms·Then they are, by definition, bots scriping the site for informations, and should start by the robots.txtby etiennebausson 2y agoThen they are, by definition, bots scriping the site for informations, and should start by the robots.txt