5 ms·

The example you've provided is confusion/complexity related to what a "character" is. If a language supports a string type and related functions that equate "c

by raiph 11y ago

The example you've provided is confusion/complexity related to what a "character" is. If a language supports a string type and related functions that equate "character" with a grapheme they'll deliver your first two invariants. With an appropriate approach to byte representations your third invariant can also be maintained.

Someone 11y ago
If you look at Swift, it uses what Apple calls "extended grapheme cluster" as its "Character" type. That means: "e" + "`" = "è" All three strings have 1 Swift Character in them; the first is not a prefix of the last. And yes, you can normalize byte representations, but then you have to give up round-tripping through strings. I maintain convinced that there is no way to do Unicode strings without drawbacks.
- raiph 11y ago
  I haven't tried Swift yet but my latest observations and guessing about your grave accent example are as follows. Based on direct cut/paste and inspection, the grave accent (in the "`" string above and here in these parens too) is codepoint 96. This is a non-combining (standalone) grave accent that is meant to be its own grapheme. Appending it to another grapheme such as "e" should not combine. The result should be an "e" followed by a visually separate grave character. The string should contain two separate graphemes and have a length of 2. I can't tell if the confusion/complexity in this case is due to some cut/paste/storage/browser bug, or Swift's handling of the special case (called "degenerate" in the Unicode standard) of an isolated combining character that hasn't yet been combined, or something else. So I read some of the Swift docs to see if that revealed anything. The Swift docs I read used the example of appending a combining grave accent (ie codepoint 768, which is NOT the same as the codepoint 96 stored in "`", but rather something more akin to "̀"), stored as a Character value, to an "e" string, stored as a String value. After appending a Character that is a combining character, the length of the new String is the same as the old one. And that's appropriate. I presume that the Swiftian character count concept, which is applicable to Swift Strings, and which is returned by .count, is inapplicable to Swift's Character values. Presumably, when a Swift Character value needs to be displayed, Swift generates a String that contains it and displays that. Further, perhaps, if the Character value is a combining character, Swift automatically prepends an appropriate base character (a space, value 32, or NBSP, or dotted circle, or some such) to ensure it becomes its own grapheme. Part of the confusion may stem from the decision in Swift to use "..." quotemarks for literals of both types. It seems you have to explicitly tell Swift that a "..." literal is a Character, not a String, if you want a Character. At first blush I like the sound of that. Alternatively, they've screwed something up.

by raiph 11y ago

Someone 11y ago

If you look at Swift, it uses what Apple calls "extended grapheme cluster" as its "Character" type. That means: "e" + "`" = "è" All three strings have 1 Swift Character in them; the first is not a prefix of the last. And yes, you can normalize byte representations, but then you have to give up round-tripping through strings. I maintain convinced that there is no way to do Unicode strings without drawbacks.

raiph 11y ago
I haven't tried Swift yet but my latest observations and guessing about your grave accent example are as follows. Based on direct cut/paste and inspection, the grave accent (in the "`" string above and here in these parens too) is codepoint 96. This is a non-combining (standalone) grave accent that is meant to be its own grapheme. Appending it to another grapheme such as "e" should not combine. The result should be an "e" followed by a visually separate grave character. The string should contain two separate graphemes and have a length of 2. I can't tell if the confusion/complexity in this case is due to some cut/paste/storage/browser bug, or Swift's handling of the special case (called "degenerate" in the Unicode standard) of an isolated combining character that hasn't yet been combined, or something else. So I read some of the Swift docs to see if that revealed anything. The Swift docs I read used the example of appending a combining grave accent (ie codepoint 768, which is NOT the same as the codepoint 96 stored in "`", but rather something more akin to "̀"), stored as a Character value, to an "e" string, stored as a String value. After appending a Character that is a combining character, the length of the new String is the same as the old one. And that's appropriate. I presume that the Swiftian character count concept, which is applicable to Swift Strings, and which is returned by .count, is inapplicable to Swift's Character values. Presumably, when a Swift Character value needs to be displayed, Swift generates a String that contains it and displays that. Further, perhaps, if the Character value is a combining character, Swift automatically prepends an appropriate base character (a space, value 32, or NBSP, or dotted circle, or some such) to ensure it becomes its own grapheme. Part of the confusion may stem from the decision in Swift to use "..." quotemarks for literals of both types. It seems you have to explicitly tell Swift that a "..." literal is a Character, not a String, if you want a Character. At first blush I like the sound of that. Alternatively, they've screwed something up.