3 ms·

Anyone familiar with Unicode development know why combining characters follow the base character? Prefix combining characters would have had two nice properties

by kps 10y ago

Anyone familiar with Unicode development know why combining characters follow the base character? Prefix combining characters would have had two nice properties: [1] Keyboard dead keys could simply generate the combining character, rather than maintain internal state, and [2] it would be possible to find the end of a combining sequence without lookahead.

Manishearth 10y ago
A bunch of characters are prepended combining marks (so lookahead doesn't quite work) 0600..0605 ; Prepend # Cf [6] ARABIC NUMBER SIGN..ARABIC NUMBER MARK ABOVE 06DD ; Prepend # Cf ARABIC END OF AYAH 070F ; Prepend # Cf SYRIAC ABBREVIATION MARK 08E2 ; Prepend # Cf ARABIC DISPUTED END OF AYAH 0D4E ; Prepend # Lo MALAYALAM LETTER DOT REPH 110BD ; Prepend # Cf KAITHI NUMBER SIGN 111C2..111C3 ; Prepend # Lo [2] SHARADA SIGN JIHVAMULIYA..SHARADA SIGN UPADHMANIYA While Unicode-specced grapheme boundaries (UAX 29) do not deal with these, Indic scripts have an "infix" combining mark that takes two regular code points on either side to form a consonant cluster. (Also, Hangul is based on conjoining code points where none of the code points is really a "combiner", they're all equally "regular" characters) I suspect the actual answer to your question is "historical accident". But, the dead key argument doesn't quite hold -- I can equally ask why keyboard dead keys aren't typed after the character.
- kps 10y ago
  > I can equally ask why keyboard dead keys aren't typed after the character. That does have an answer: keyboards predate electronics. A mechanical typewriter key press slams a type bar into the ribbon, and on the way back, catches a mechanism that releases the carriage (which is pulled along by a spring) for one column. A dead key is very simple; it just omits the catch. A postfix dead key would be more complicated, since it would have to add an early backspace action, and harder to press, since backspacing pulls against the carriage spring.
  - Manishearth 10y ago
    This is pretty cool and interesting. Thanks!
derefr 10y ago
Unicode is in large part about being resilient in the face of partial corruption or https://en.wikipedia.org/wiki/Bit_slip https://en.wikipedia.org/wiki/Bit_slip. So the answer probably lies in what the Unicode codepoint lexing algorithm must do to disregard "dependent" combining characters, if it hits a corrupted base character.

by kps 10y ago

Manishearth 10y ago

A bunch of characters are prepended combining marks (so lookahead doesn't quite work) 0600..0605 ; Prepend # Cf [6] ARABIC NUMBER SIGN..ARABIC NUMBER MARK ABOVE 06DD ; Prepend # Cf ARABIC END OF AYAH 070F ; Prepend # Cf SYRIAC ABBREVIATION MARK 08E2 ; Prepend # Cf ARABIC DISPUTED END OF AYAH 0D4E ; Prepend # Lo MALAYALAM LETTER DOT REPH 110BD ; Prepend # Cf KAITHI NUMBER SIGN 111C2..111C3 ; Prepend # Lo [2] SHARADA SIGN JIHVAMULIYA..SHARADA SIGN UPADHMANIYA While Unicode-specced grapheme boundaries (UAX 29) do not deal with these, Indic scripts have an "infix" combining mark that takes two regular code points on either side to form a consonant cluster. (Also, Hangul is based on conjoining code points where none of the code points is really a "combiner", they're all equally "regular" characters) I suspect the actual answer to your question is "historical accident". But, the dead key argument doesn't quite hold -- I can equally ask why keyboard dead keys aren't typed after the character.

kps 10y ago
> I can equally ask why keyboard dead keys aren't typed after the character. That does have an answer: keyboards predate electronics. A mechanical typewriter key press slams a type bar into the ribbon, and on the way back, catches a mechanism that releases the carriage (which is pulled along by a spring) for one column. A dead key is very simple; it just omits the catch. A postfix dead key would be more complicated, since it would have to add an early backspace action, and harder to press, since backspacing pulls against the carriage spring.
- Manishearth 10y ago
  This is pretty cool and interesting. Thanks!

derefr 10y ago

Unicode is in large part about being resilient in the face of partial corruption or https://en.wikipedia.org/wiki/Bit_slip https://en.wikipedia.org/wiki/Bit_slip. So the answer probably lies in what the Unicode codepoint lexing algorithm must do to disregard "dependent" combining characters, if it hits a corrupted base character.