4 ms·First comment on OP says they ignore robots.txtby RyJones 13y agoFirst comment on OP says they ignore robots.txt