3 ms·

Nothing, I don't make assumptions. That's something you do.

• eesmith 4y ago
  You assumed people don't give a crap about robots.txt, otherwise you wouldn't have been surprised.