3 ms·

I considered that. It seems like the only solution (to me). When the <meta> tag for a canonical tag is available it seems pretty easy to solve that way. I'm re

I considered that. It seems like the only solution (to me). When the <meta> tag for a canonical tag is available it seems pretty easy to solve that way. I'm really curious how Google (and other search engines) solve this problem. Do they maintain a massive list of query parameters to ignore? JSESSIONID and the like are easy enough, but there are many others. Many pages don't define a canonical URL with a <meta> tag either.

byoung2 16y ago
I'm really curious how Google (and other search engines) solve this problem. Do they maintain a massive list of query parameters to ignore? JSESSIONID and the like are easy enough, but there are many others I would imagine when google crawls a page, they analyze the content and compare it to other pages with similar URLs. If the content is identical to another page with the same URL, they look for a canonical URL and index using that. If not, they pick one URL to represent the many identical pages. Over time crawling a site, they would know which parameters to ignore.