2 ms·

> convert Unicode into and out of a simpler format to use internally But what simpler format? If it supports all of Unicode it's not simpler. If it doesn't, t

> convert Unicode into and out of a simpler format to use internally But what simpler format? If it supports all of Unicode it's not simpler. If it doesn't, then it's missing necessary features entire languages rely on. And missing emoji. I don't understand what benefit you're trying to provide.

Nevermark 2y ago
Unicode includes characters built from multiple characters. This means questions (functions) like “how many characters do I have”, and indexing in general, are not as straight forward as they could be. So storing each visual character’s character group together, as a single element, results in a simpler interface from a programming standpoint. Not necessarily as efficient, since either indirection or larger character footprints (or both) are required, but simpler. This is definitely the route I would go, while implementing other complexity. After that, possibly undoing this “simplicity optimization”, would be a small price to pay for the development convenience.
Findecanor 2y ago
Simpler for the text editor logic, not necessarily simpler for the encoder/decoder. > If it supports all of Unicode it's not simpler. The point is that it doesn't. The Unicode that it does not handle should get marked as "illegal character", rendered as boxes and not get mangled unless the user messes with it. Support for more of Unicode could be added incrementally, and in a single subsystem instead of everywhere in the text editor.
- crazygringo 2y ago
  I still don't understand what you're arguing for. >> If it supports all of Unicode it's not simpler. > The point is that it doesn't. But then it doesn't work. Why would you not want to support all of Unicode? > The Unicode that it does not handle should get marked as "illegal character", rendered as boxes and not get mangled unless the user messes with it. That's what already happens. We already show boxes when the codepoint is unknown or there's no glyph in the font. Their codepoints are preserved. > Support for more of Unicode could be added incrementally, and in a single subsystem instead of everywhere in the text editor. It already is. New OS's get upgraded to new Unicode versions. Text editors don't usually have to change a thing. I don't know what problem you're trying to solve here.

Nevermark 2y ago

Unicode includes characters built from multiple characters. This means questions (functions) like “how many characters do I have”, and indexing in general, are not as straight forward as they could be. So storing each visual character’s character group together, as a single element, results in a simpler interface from a programming standpoint. Not necessarily as efficient, since either indirection or larger character footprints (or both) are required, but simpler. This is definitely the route I would go, while implementing other complexity. After that, possibly undoing this “simplicity optimization”, would be a small price to pay for the development convenience.

Findecanor 2y ago

Simpler for the text editor logic, not necessarily simpler for the encoder/decoder. > If it supports all of Unicode it's not simpler. The point is that it doesn't. The Unicode that it does not handle should get marked as "illegal character", rendered as boxes and not get mangled unless the user messes with it. Support for more of Unicode could be added incrementally, and in a single subsystem instead of everywhere in the text editor.

crazygringo 2y ago
I still don't understand what you're arguing for. >> If it supports all of Unicode it's not simpler. > The point is that it doesn't. But then it doesn't work. Why would you not want to support all of Unicode? > The Unicode that it does not handle should get marked as "illegal character", rendered as boxes and not get mangled unless the user messes with it. That's what already happens. We already show boxes when the codepoint is unknown or there's no glyph in the font. Their codepoints are preserved. > Support for more of Unicode could be added incrementally, and in a single subsystem instead of everywhere in the text editor. It already is. New OS's get upgraded to new Unicode versions. Text editors don't usually have to change a thing. I don't know what problem you're trying to solve here.