3 ms·
I think you got the wrong RFC number: http://tools.ietf.org/rfc/rfc3938.txt http://tools.ietf.org/rfc/rfc3938.txt No mention of robots.txt, nor a section 6.
by nostrademons 16y ago
I think you got the wrong RFC number:
http://tools.ietf.org/rfc/rfc3938.txt http://tools.ietf.org/rfc/rfc3938.txt
No mention of robots.txt, nor a section 6.
- jfr 16y agoSorry, the correct number was 3986. RFC 3986 - Uniform Resource Identifier (URI): Generic Syntax