3 ms·
This post makes no sense to me. Are you saying that we should still use character encodings rather than UTF encodings, or are you saying that we shouldn't assu
by sambeau 15y ago
This post makes no sense to me.
Are you saying that we should still use character encodings rather than UTF encodings, or are you saying that we shouldn't assume that raw text is ASCII, or are you saying something else?
Unicode is, essentially, nothing unless it is encoded. When you encode it you must decide whether to use one byte, two bytes, three bytes or four bytes.
Only UTF-8 and UTF-32 are really big enough to hold the world's characters. everything else is a fudge.
ASCII was never Anglo-Saxon: it was always American. COBOL is a red herring here, too. ASCII was all about teleprinters and was a clever use of 8-Bits for its time. Bell labs invented ASCII and Bell labs invented UTF-8.
UTF-8 is much better than reasonable. It is a compact way to represent Unicode while preventing the western world from having to re-encode every text document. That's a lot of good news the internet.
Are you saying that we shouldn't ignore other charsets as they are still valid Unicode? If so I agree up to a point, the point being that there is no longer any need to have any other unicode encoding other than UTF-8. If you need to access your local characters as a byte array: choose your internal encoding and translate, do your magic, and then spit out UTF-8 again then we can all simply read the same documents without the need for over-complexity.
- finnw 15y ago> Only UTF-8 and UTF-32 are really big enough to hold the world's characters So is GB18030. Edit: and UTF-16, of course (just don't confuse it with UCS-2)
- sambeau 15y agoI'm guilty of using UTF16 when I really mean 16 bit unicode arrays. I see that now. But it does beg the question why would you use UTF-16? Yes if you have more than 512 characters in your common script then OK I can see it might make sense, but not much. UTF-8 will still average out in a reasonable way.
- jpablo 15y agoBecause it's the standard in the Windows API.
- jpablo 15y agoIt makes sense to me. His point is that calling UTF-8 "Unicode" is wrong, that just confuses people. Windows programmers usually also call UTF-16 "Unicode" which is also wrong. Unicode is not a text encoding, it's a standard that assigns a number to every character.Saying that a particular text is "Unicode" doesn't give any info on how to decode it, we should just say that a given text is UTF-7,8,16,32, etc.