4 ms·

They absolutely are.

• razakel 9mo ago
  Of course they are. Had to block anything at work coming from one certain company because it wasn't respecting robots.txt and the bill was just getting silly.