3 ms·
The sort of tarpit you're talking about wouldn't even affect this link validator. You really think Stack Exchange should have given up on validating links beca
by rsofaer 14y ago
The sort of tarpit you're talking about wouldn't even affect this link validator. You really think Stack Exchange should have given up on validating links because Github's robots.txt has:
User-agent: *
Disallow: /
in it?
- lazugod 14y agoThey could ask for Github's permission.
- JasonPunyon 14y agoYeah, we did. http://meta.stackoverflow.com/a/132677/6212 http://meta.stackoverflow.com/a/132677/6212