4 ms·
Even if something new is UTF-8, you'd basically have to guarantee that it never interfaces with anything old and/or broken to ensure the data survives intact. I
by ygra 9y ago
Even if something new is UTF-8, you'd basically have to guarantee that it never interfaces with anything old and/or broken to ensure the data survives intact. I've recently received a package where the ö in my name was written as ̦ which I've never seen before. Things go wrong in the most unexpected places and the sad thing is that even if you use UTF-8 for text storage, you still have to know what you can do and how to do it to not mangle it.
There are a lot of Unicode-hostile environments out there. Java is old enough to always require explicit encoding declaration for pretty much any tool ... compiler, documentation generator, etc. Forget it at any one point and you get garbage. Reading or writing text files should always make the encoding explicit, but rarely does so. C#'s string methods all support, but don't require, a Culture parameter, without which you're practically guaranteed to do things like case conversion, or substring searches wrong in the general case. There was an awesome and long answer by tchrist on SO once about what the Perl boilerplate is to properly support Unicode for many or most circumstances (it's complicated and long and I doubt many people are going those lengths).
Point being, even when using something that supports Unicode well, the programmer still has to care, simply because text and language are messy things and it simply isn't possible to have a magic bullet that does everything right.