4 ms·Would companies start adding Datanyze to their robots.txt and would Datanyze respect that?by pbreit 13y agoWould companies start adding Datanyze to their robots.txt and would Datanyze respect that?