4 ms·
This is interesting and unexpected if true. My only thought is that virtually all "serious" sites tend to have robots.txt, and so not having it indicates a hig
by crazygringo 9mo ago
This is interesting and unexpected if true.
My only thought is that virtually all "serious" sites tend to have robots.txt, and so not having it indicates a high likelihood of spam.
- franze 9mo agonot true