3 ms·

You are much better off with doing the following: (1) All responses for non-canonical URLs are 301 redirects to the canonical URLs, (2) Your website will never

You are much better off with doing the following: (1) All responses for non-canonical URLs are 301 redirects to the canonical URLs, (2) Your website will never link to a resource using a URL other than its canonical one, (3) you encourage people to link to pages on your site using the canonical URLs. This way your site will be very cache-friendly while still being usable. Also, all search engines will be able to understand your site without any proprietary extensions (a.k.a. "standards" at Google, apparently) being needed.

wmf 18y ago
I generally agree with that approach, but Google gives an example of pages with query strings that need to have different URLs because they are subtly different, but not in a way that search engines need to care about. OTOH, Google's wiki example is bogus; people have been telling MediaWiki that they should be using 301s for years but they just won't. This workaround just encourages them to never fix it.
- zepolen 18y ago
  It's not just one example, you can 'hurt' a website simply by making a huge list of links with arbritrary query string garbage so that google picks it up, eg: http://domain.com/?dupcontent http://domain.com/?dupcontent http://domain.com/?blabla http://domain.com/?blabla What an app should really do is validate the arguments in the query string, remove any invalid ones, then issue a 301 redirect to the proper url. For example: http://www.google.com/search?q=someterm&unknown_variable=nothing http://www.google.com/search?q=someterm&unknown_variable... redirects to http://www.google.com/search?q=someterm http://www.google.com/search?q=someterm Of course that's like getting everyone's CSS to validate correctly :) Edit: It is also why I prefer to sort my query string so that it can be deterministic and always be the same no matter what order the args are in.
  - briansmith 18y ago
    That kind of attack won't work on a website that does a 301 redirect from all non-canonical URLs to the canonical ones.

wmf 18y ago

I generally agree with that approach, but Google gives an example of pages with query strings that need to have different URLs because they are subtly different, but not in a way that search engines need to care about. OTOH, Google's wiki example is bogus; people have been telling MediaWiki that they should be using 301s for years but they just won't. This workaround just encourages them to never fix it.

zepolen 18y ago
It's not just one example, you can 'hurt' a website simply by making a huge list of links with arbritrary query string garbage so that google picks it up, eg: http://domain.com/?dupcontent http://domain.com/?dupcontent http://domain.com/?blabla http://domain.com/?blabla What an app should really do is validate the arguments in the query string, remove any invalid ones, then issue a 301 redirect to the proper url. For example: http://www.google.com/search?q=someterm&unknown_variable=nothing http://www.google.com/search?q=someterm&unknown_variable... redirects to http://www.google.com/search?q=someterm http://www.google.com/search?q=someterm Of course that's like getting everyone's CSS to validate correctly :) Edit: It is also why I prefer to sort my query string so that it can be deterministic and always be the same no matter what order the args are in.
- briansmith 18y ago
  That kind of attack won't work on a website that does a 301 redirect from all non-canonical URLs to the canonical ones.