3 ms·
Totally agreeing on the benefits. But remember one thing: You english speaking people out there have it easy because of points 1-3 in the article. This stops b
by pilif 16y ago
Totally agreeing on the benefits. But remember one thing: You english speaking people out there have it easy because of points 1-3 in the article.
This stops being the case the moment you are dealing with any non-ascii character. At that point, some assumptions stop being valid, like the fact that it stops being the case that every document is a valid UTF-8 encoded document.
If you treat arbitrary encoded data as UTF-8, depending on your environment, you will get thrown exceptions at or you will see question marks in various designs all over the place.
Combine this with the fact that browsers sometimes are not quite sure of what they are doing:
I've seen them sending latin1 but telling the server it's utf-8 or the other way around.
The rails people tried to detect utf-8-ness using the snowman and lately a checkmark (http://railssnowman.info/ http://railssnowman.info/).
Finally, keep in mind that it's impossible to accurately detect the encoding if it isn't utf-8 without actually doing language analysis.
Soon, you'll notice that utf-8 isn't magically solving problems.
It's funny how many english speaking people talking to english audiences think that slapping a "; charset=utf-8" to their content-type headers suddenly makes their site utf-8 compliant.
As long as their content is 7 bit english and their users send data in 7 bit english, they could as well just have left the charset alone or set it to ASCII as ASCII = utf-8 as long as the first bit isn't set.
The application I'm maintaining, while being multi lingual, is, thankfully, targeting countries with languages that can be expressed in latin-1, so that's what we are (still) using.
I made various attempts at going utf-8, but in the end, between browsers lying and external third-party APIs still insisting on latin1, these efforts never bore fruit.
- pornel 16y agoStill, I prefer site to have naïve charset=UTF-8 slapped that at least allows me to use non-ASCII characters mostly correctly most of the time, than to deal with ISO-8859-1 site (like Amazon, grr) which leaves me no choice but to omit or transliterate most characters.
- rimantas 16y agoSure, saving file as UTF8 before specifying encoding as UTF8 is mission impossible.
- qntm 16y agoI don't see any solution to character encoding problems while web browsers actively lie about what they are sending. But that's not a problem with UTF-8, or even with any specific encoding.
- pilif 16y agoof course not. I was just saying that the fact that every ASCII document is also a UTF-8 document only makes life easier for people dealing with ASCII. The moment you leave that safe area, all the usual problems will come and haunt you.