> I wonder how many of these sites are just reporting UTF-8 or something because their web server defaults to it, and not actually encoding special characters p

> I wonder how many of these sites are just reporting UTF-8 or something because their web server defaults to it, and not actually encoding special characters properly? I think they are measuring the encoding that is picked before adding a page to the index. Which would be a combination of explicit information (headers, xml and html metadata) and heuristics (which may fail, but are useful when the explicit information is missing or — even though ignoring explicit metadata is bad — obviously incorrect). It also seems they are looking for a subset encoding once they have the metadata; the posts describe explicitly labelling ASCII when the contents are within that subset. But their methodology has changed from the previous two posts: latin above ascii in 2008 didn't exist in http://googleblog.blogspot.com/2010/01/unicode-nearing-50-of-web.html http://googleblog.blogspot.com/2010/01/unicode-nearing-50-of... or http://googleblog.blogspot.com/2008/05/moving-to-unicode-51.html http://googleblog.blogspot.com/2008/05/moving-to-unicode-51.... .