3 ms·
Why? You're not a web crawler.
by stuartmemo 13y ago
Why? You're not a web crawler.
- MostAwesomeDude 13y agoThen I would humbly and politely suggest that the website use a header like `Vary: User-Agent` in its response to indicate that different user agents are given different responses. Alternatively, use a 300 Multiple Choices status code to present the two different versions of the site: The crawlable version, and the JS-encumbered version. I understand that RFC 2616 does not forbid this technique, but something is not right simply because it is not wrong, and it's deceptive in either case. Do what is right and good, not what is merely not wrong.