4 ms·

Emoji is just the easiest way to present these problems to an English-reading audience. For the several billion people reading and writing non-latin scripts whe

by duped 2y ago

Emoji is just the easiest way to present these problems to an English-reading audience. For the several billion people reading and writing non-latin scripts where the "edge cases" are important for legibility/correctness, yes, we need text editors that can handle them.

dsego 2y ago
I'm proposing it can be done at one layer above or in a different abstraction. There seems to be a point of diminishing returns, I mean if your text editor doesn't handle knotted cords from the inca empire why even bother.
- duped 2y ago
  The native languages of roughly half the global population isn't exactly "knotted cords from the inca empire." As for whether it can be in a layer below the text editor... probably not, but even then, someone has to write it and talk about why determining "how many glyphs wide is this sequence of bytes" is an interesting question with a complex answer worthy of study. And today the answer to that question means "maintain a very large and dynamic dataset of the world's written languages" which makes it unpalatable to "low level" software, like the standard libraries of every PL that I'm aware of.
dghlsakjg 2y ago
We don’t need text editors to handle all character sets in the same way that I don’t need code editors to be able to parse every language without changing settings. If Arabic text editors don’t have the cursor behave the same as English ones, that’s fine. If I press backspace in a Hebrew text editor I probably want it to delete the character to the right. The behavior doesn’t need to be universal
- philistine 2y ago
  What is a Hebrew text editor? Is VS Code only for anglophones or something?
- groby_b 2y ago
  Except e.g. in Hebrew, you almost certainly have bidirectional text. E.g. numbers in a text still follow LTR. So if you press backspace, you don't always want to delete the text to the right. Now add to that the fact that nobody wants to build a new editor for every single language, and there you go - all of that gets rolled into a common approach. It's hard to impossible to get right. (Because what you want also depends on context, not just language - URLs demand different behaviors with BiDi than text does) Even if we wanted to build different editors, as soon as you start quoting things not from your preferred environment, guess what, you have mixed environments and need to start thinking about BiDi It's a gargantuan mess. Nobody is really getting it right, we're just trying to get closer and closer. (And if you want to see folks in the field pulling out their hair, mention you'd like Boustrophedon writing ;) Source: UAX 9 and UAX 29 were printouts on my desk for several years, for very good reasons. I still have unsolved problems with BiDi text, I've just given up.

by duped 2y ago

dsego 2y ago

I'm proposing it can be done at one layer above or in a different abstraction. There seems to be a point of diminishing returns, I mean if your text editor doesn't handle knotted cords from the inca empire why even bother.

duped 2y ago
The native languages of roughly half the global population isn't exactly "knotted cords from the inca empire." As for whether it can be in a layer below the text editor... probably not, but even then, someone has to write it and talk about why determining "how many glyphs wide is this sequence of bytes" is an interesting question with a complex answer worthy of study. And today the answer to that question means "maintain a very large and dynamic dataset of the world's written languages" which makes it unpalatable to "low level" software, like the standard libraries of every PL that I'm aware of.

dghlsakjg 2y ago

We don’t need text editors to handle all character sets in the same way that I don’t need code editors to be able to parse every language without changing settings. If Arabic text editors don’t have the cursor behave the same as English ones, that’s fine. If I press backspace in a Hebrew text editor I probably want it to delete the character to the right. The behavior doesn’t need to be universal

philistine 2y ago
What is a Hebrew text editor? Is VS Code only for anglophones or something?
groby_b 2y ago
Except e.g. in Hebrew, you almost certainly have bidirectional text. E.g. numbers in a text still follow LTR. So if you press backspace, you don't always want to delete the text to the right. Now add to that the fact that nobody wants to build a new editor for every single language, and there you go - all of that gets rolled into a common approach. It's hard to impossible to get right. (Because what you want also depends on context, not just language - URLs demand different behaviors with BiDi than text does) Even if we wanted to build different editors, as soon as you start quoting things not from your preferred environment, guess what, you have mixed environments and need to start thinking about BiDi It's a gargantuan mess. Nobody is really getting it right, we're just trying to get closer and closer. (And if you want to see folks in the field pulling out their hair, mention you'd like Boustrophedon writing ;) Source: UAX 9 and UAX 29 were printouts on my desk for several years, for very good reasons. I still have unsolved problems with BiDi text, I've just given up.