3 ms·Respecting robots.txt would mean not saving pages blocked by robots.txt. Not just hiding.by pseudalopex 5y agoRespecting robots.txt would mean not saving pages blocked by robots.txt. Not just hiding.