3 ms·Yup. Robots.txt was a don't-swamp-me thing.by LorenPechtel 1y agoYup. Robots.txt was a don't-swamp-me thing.