4 ms·

That's the point of the post, we should not have to know anything about encoding. It should just work.

by dan_sim 16y ago

crazydiamond 16y ago
I think there are trickier issues which a programming language has to deal with. For example, concatenating 2 strings using different encoding. Conversion is required. Specifying encoding is required. And btw, here there is a loss since all character sets are not fully covered even in UTF-16 (iirc). I am trying to recall- maybe Matz gave a details reply somewhere on the net.
- chbarts 16y ago
  > And btw, here there is a loss since all character sets are not fully covered even in UTF-16 (iirc). UTF-16 covers everything in Unicode if you deal with surrogate pairs. UCS-2 does not: It's limited to the Basic Multilingual Plane, which captures a lot but not everything. In particular, East Asian languages are only partially covered if you only deal with UCS-2 (or UTF-16 without surrogate pairs). UTF-8 covers everything without surrogate pairs or any other hack: It's variable-width, and somewhat more complex to decode, but it's a flat encoding, as opposed to the segmentation enforced by surrogate pairs. Now, some things just aren't in Unicode (yet). There's very little you can reasonably do about that, unless you happen to know how to convert to and from an encoding that does have them. Fortunately, Unicode covers damn near everything these days.
- sid0 16y ago
  > And btw, here there is a loss since all character sets are not fully covered even in UTF-16 (iirc) If they aren't covered in UTF-16, they wouldn't be covered in UTF-8 or UCS-4 either. All modern Unicode encodings (i.e. not UCS-2) can encode exactly the same data. I'd be curious to know which character sets Unicode doesn't cover yet a different encoding system does.
chbarts 16y ago
From the fortune files: When someone says "I want a programming language in which I need only say what I wish done," give him a lollipop.
brazzy 16y ago
The world does not care about what you think it should be like.

That's the point of the post, we should not have to know anything about encoding. It should just work.

by dan_sim 16y ago

That's the point of the post, we should not have to know anything about encoding. It should just work.

I think there are trickier issues which a programming language has to deal with. For example, concatenating 2 strings using different encoding. Conversion is required. Specifying encoding is required. And btw, here there is a loss since all character sets are not fully covered even in UTF-16 (iirc). I am trying to recall- maybe Matz gave a details reply somewhere on the net.

chbarts 16y ago
> And btw, here there is a loss since all character sets are not fully covered even in UTF-16 (iirc). UTF-16 covers everything in Unicode if you deal with surrogate pairs. UCS-2 does not: It's limited to the Basic Multilingual Plane, which captures a lot but not everything. In particular, East Asian languages are only partially covered if you only deal with UCS-2 (or UTF-16 without surrogate pairs). UTF-8 covers everything without surrogate pairs or any other hack: It's variable-width, and somewhat more complex to decode, but it's a flat encoding, as opposed to the segmentation enforced by surrogate pairs. Now, some things just aren't in Unicode (yet). There's very little you can reasonably do about that, unless you happen to know how to convert to and from an encoding that does have them. Fortunately, Unicode covers damn near everything these days.
sid0 16y ago
> And btw, here there is a loss since all character sets are not fully covered even in UTF-16 (iirc) If they aren't covered in UTF-16, they wouldn't be covered in UTF-8 or UCS-4 either. All modern Unicode encodings (i.e. not UCS-2) can encode exactly the same data. I'd be curious to know which character sets Unicode doesn't cover yet a different encoding system does.

chbarts 16y ago

From the fortune files: When someone says "I want a programming language in which I need only say what I wish done," give him a lollipop.

brazzy 16y ago

The world does not care about what you think it should be like.