2 ms·Wouldn't exclusion via robots.txt be appropriate in this case?by JoachimSchipper 17y agoWouldn't exclusion via robots.txt be appropriate in this case?