6 ms·Or searchyc could simply ignore robots.txtby mkull 17y agoOr searchyc could simply ignore robots.txt