4 ms·

It's good to know that it's on the increase. Still, character encoding seems to be something that is understood by very few people... I wonder how many of these

It's good to know that it's on the increase. Still, character encoding seems to be something that is understood by very few people... I wonder how many of these sites are just reporting UTF-8 or something because their web server defaults to it, and not actually encoding special characters properly? It's certainly very easy to do - I had the problem a little while back setting up a little web app where the web server and MySQL database were both UTF-8, but the db connection was defaulting to ISO-8859-1 or something, causing all sorts of issues with curly quotes etc.

wladimir 15y ago
The great thing is that one now only has to explain to people to use UTF-8 in all cases, which is preferable to having to fool around with a lot of language-specific encodings. And it helps that almost all editors, these days, also default to UTF-8 encoding. So you can just copy/paste special characters and it works...
obtu 15y ago
> I wonder how many of these sites are just reporting UTF-8 or something because their web server defaults to it, and not actually encoding special characters properly? I think they are measuring the encoding that is picked before adding a page to the index. Which would be a combination of explicit information (headers, xml and html metadata) and heuristics (which may fail, but are useful when the explicit information is missing or — even though ignoring explicit metadata is bad — obviously incorrect). It also seems they are looking for a subset encoding once they have the metadata; the posts describe explicitly labelling ASCII when the contents are within that subset. But their methodology has changed from the previous two posts: latin above ascii in 2008 didn't exist in http://googleblog.blogspot.com/2010/01/unicode-nearing-50-of-web.html http://googleblog.blogspot.com/2010/01/unicode-nearing-50-of... or http://googleblog.blogspot.com/2008/05/moving-to-unicode-51.html http://googleblog.blogspot.com/2008/05/moving-to-unicode-51.... .
- justincormack 15y ago
  Yes I wondered this. I would love to see figures for incorrectly labelled on that chart.
VMG 15y ago
Java is among the last holdouts for Latin-1-encoding - you still have to jump through hoops with Spring MVC to properly send UTF-8 encoded responses http://stackoverflow.com/questions/3616359 http://stackoverflow.com/questions/3616359 Also the encoding for .properties files is specified as Latin-1 http://en.wikipedia.org/wiki/.properties http://en.wikipedia.org/wiki/.properties
drhowarddrfine 15y ago
All browsers use UTF by default and all web pages should be serving UTF-8: http://hsivonen.iki.fi/accept-charset/ http://hsivonen.iki.fi/accept-charset/
lmm 15y ago
Mysql's character encoding is pretty hilarious. I love how its "latin1" is nothing of the sort (it's actually cp1252 with eight characters shuffled around).

wladimir 15y ago

The great thing is that one now only has to explain to people to use UTF-8 in all cases, which is preferable to having to fool around with a lot of language-specific encodings. And it helps that almost all editors, these days, also default to UTF-8 encoding. So you can just copy/paste special characters and it works...

obtu 15y ago

> I wonder how many of these sites are just reporting UTF-8 or something because their web server defaults to it, and not actually encoding special characters properly? I think they are measuring the encoding that is picked before adding a page to the index. Which would be a combination of explicit information (headers, xml and html metadata) and heuristics (which may fail, but are useful when the explicit information is missing or — even though ignoring explicit metadata is bad — obviously incorrect). It also seems they are looking for a subset encoding once they have the metadata; the posts describe explicitly labelling ASCII when the contents are within that subset. But their methodology has changed from the previous two posts: latin above ascii in 2008 didn't exist in http://googleblog.blogspot.com/2010/01/unicode-nearing-50-of-web.html http://googleblog.blogspot.com/2010/01/unicode-nearing-50-of... or http://googleblog.blogspot.com/2008/05/moving-to-unicode-51.html http://googleblog.blogspot.com/2008/05/moving-to-unicode-51.... .

justincormack 15y ago
Yes I wondered this. I would love to see figures for incorrectly labelled on that chart.

VMG 15y ago

Java is among the last holdouts for Latin-1-encoding - you still have to jump through hoops with Spring MVC to properly send UTF-8 encoded responses http://stackoverflow.com/questions/3616359 http://stackoverflow.com/questions/3616359 Also the encoding for .properties files is specified as Latin-1 http://en.wikipedia.org/wiki/.properties http://en.wikipedia.org/wiki/.properties

drhowarddrfine 15y ago

All browsers use UTF by default and all web pages should be serving UTF-8: http://hsivonen.iki.fi/accept-charset/ http://hsivonen.iki.fi/accept-charset/

lmm 15y ago

Mysql's character encoding is pretty hilarious. I love how its "latin1" is nothing of the sort (it's actually cp1252 with eight characters shuffled around).