5 ms·

Do we actually need text editing support all conceivable edge cases? I guess I don't care how the cursor behaves as long as it's mostly predictable and consiste

by dsego 2y ago

Do we actually need text editing support all conceivable edge cases? I guess I don't care how the cursor behaves as long as it's mostly predictable and consistent. Users would adapt, I must not use or care about skinned emojis much (seems like a fad anyway), but I use sublime text and wouldn't really care how it handles them at all.

bigstrat2003 2y ago
I agree. Specifically, emojis are not text and I don't really care what edge cases they introduce. It's neat if someone wants to put the work in, but that's not the meat of "text editing" and isn't really worth worrying about.
- samatman 2y ago
  Emoji are text in all the ways that matter to a programmer. There are codepoints in Unicode which aren't textual, but emoji are not among them. Emoji don't introduce any edge cases of their own. If your software can't join a skin tone to a thumbs up, it also will fall apart on composing many scripts used by hundreds of millions of people. So if you really want to get all kids-these-days about emoji, go ahead, but support Unicode properly. You'll get correct emoji handling as a consequence of that anyway, and you can still signal whatever message you're intending to send here about yourself in some other way.
  - fsckboy 2y ago
    ASCII is text in all the ways that matter to a programmer.
- keybored 2y ago
  That doesn’t change anything as long as there are scripts that use the same building blocks that emojis use.
duped 2y ago
Emoji is just the easiest way to present these problems to an English-reading audience. For the several billion people reading and writing non-latin scripts where the "edge cases" are important for legibility/correctness, yes, we need text editors that can handle them.
- dsego 2y ago
  I'm proposing it can be done at one layer above or in a different abstraction. There seems to be a point of diminishing returns, I mean if your text editor doesn't handle knotted cords from the inca empire why even bother.
  - duped 2y ago
    The native languages of roughly half the global population isn't exactly "knotted cords from the inca empire." As for whether it can be in a layer below the text editor... probably not, but even then, someone has to write it and talk about why determining "how many glyphs wide is this sequence of bytes" is an interesting question with a complex answer worthy of study. And today the answer to that question means "maintain a very large and dynamic dataset of the world's written languages" which makes it unpalatable to "low level" software, like the standard libraries of every PL that I'm aware of.
- dghlsakjg 2y ago
  We don’t need text editors to handle all character sets in the same way that I don’t need code editors to be able to parse every language without changing settings. If Arabic text editors don’t have the cursor behave the same as English ones, that’s fine. If I press backspace in a Hebrew text editor I probably want it to delete the character to the right. The behavior doesn’t need to be universal
  - philistine 2y ago
    What is a Hebrew text editor? Is VS Code only for anglophones or something?
  - 2y ago
biwills 2y ago
I'd say it's less about covering all edge cases and more about showing that text editors are insanely complex, and it doesn't take long to find edge cases that text editors with 10s/100s of millions of users have.
tajpulo 2y ago
And which blocks of Unicode exactly would you exclude from support and what complexity reduction do you exactly expect from it? In my experience, the complexity comes from the reality of how we write in different writing systems. I think this is inevitable.
yen223 2y ago
Even if you could pretend that everything is ASCII, text editing is still a very complex subject once you get into the weeds. You could probably write a book just on handling text overflow
Analemma_ 2y ago
Someday I want to write a lengthy essay on the software-specific variant of the "Seeing Like a State" problem where certain programmers-- angry that the Real World is actually quite complicated-- demand that the world be made simpler to accommodate their wishes for an easier job and cleaner code architecture. This comment is a great example of this phenomenon, as are stupid proposals like "abolish time zones and make everyone live on UTC because I don't like writing time-and-date code".
- dsego 2y ago
  But why not make the world simpler? Eg. let's simplify those writing systems that are causing issues. Not everything needs to be encoded as text strings. Good thing you mention UTC, because guess what, time zones are a simplification, real local time is on a continuous spectrum.
  - Analemma_ 2y ago
    Because that's authoritarian and ghoulish? The thought of forcibly changing (or eliminating) a writing system that may have been in use by millions of people for centuries with no issues, to satisfy the demands of a bunch of web developers who don't want to add some more if-else branches to their text rendering code, gives me the heebie-jeebies. And it wouldn't even save most programmers any effort: for 99.9% of us, this stuff is all abstracted away by libraries anyway. I suspect you might be over-indexing on the specific example of emojis and skin tone modifiers that the article used, but on the Unicode implementation level that's just the same modifier characters that several real languages need to use to be encoded properly. It's not a useless frivolity.
    - dsego 2y ago
      Not sure why writing systems couldn't be improved and simplified for the digital age. It would reduce software complexity and be more efficient and economical.
      - MonkeyClub 2y ago

by dsego 2y ago

bigstrat2003 2y ago

I agree. Specifically, emojis are not text and I don't really care what edge cases they introduce. It's neat if someone wants to put the work in, but that's not the meat of "text editing" and isn't really worth worrying about.

samatman 2y ago
Emoji are text in all the ways that matter to a programmer. There are codepoints in Unicode which aren't textual, but emoji are not among them. Emoji don't introduce any edge cases of their own. If your software can't join a skin tone to a thumbs up, it also will fall apart on composing many scripts used by hundreds of millions of people. So if you really want to get all kids-these-days about emoji, go ahead, but support Unicode properly. You'll get correct emoji handling as a consequence of that anyway, and you can still signal whatever message you're intending to send here about yourself in some other way.
- fsckboy 2y ago
  ASCII is text in all the ways that matter to a programmer.
keybored 2y ago
That doesn’t change anything as long as there are scripts that use the same building blocks that emojis use.

duped 2y ago

Emoji is just the easiest way to present these problems to an English-reading audience. For the several billion people reading and writing non-latin scripts where the "edge cases" are important for legibility/correctness, yes, we need text editors that can handle them.

dsego 2y ago
I'm proposing it can be done at one layer above or in a different abstraction. There seems to be a point of diminishing returns, I mean if your text editor doesn't handle knotted cords from the inca empire why even bother.
- duped 2y ago
  The native languages of roughly half the global population isn't exactly "knotted cords from the inca empire." As for whether it can be in a layer below the text editor... probably not, but even then, someone has to write it and talk about why determining "how many glyphs wide is this sequence of bytes" is an interesting question with a complex answer worthy of study. And today the answer to that question means "maintain a very large and dynamic dataset of the world's written languages" which makes it unpalatable to "low level" software, like the standard libraries of every PL that I'm aware of.
dghlsakjg 2y ago
We don’t need text editors to handle all character sets in the same way that I don’t need code editors to be able to parse every language without changing settings. If Arabic text editors don’t have the cursor behave the same as English ones, that’s fine. If I press backspace in a Hebrew text editor I probably want it to delete the character to the right. The behavior doesn’t need to be universal
- philistine 2y ago
  What is a Hebrew text editor? Is VS Code only for anglophones or something?
- 2y ago

biwills 2y ago

I'd say it's less about covering all edge cases and more about showing that text editors are insanely complex, and it doesn't take long to find edge cases that text editors with 10s/100s of millions of users have.

tajpulo 2y ago

And which blocks of Unicode exactly would you exclude from support and what complexity reduction do you exactly expect from it? In my experience, the complexity comes from the reality of how we write in different writing systems. I think this is inevitable.

yen223 2y ago

Even if you could pretend that everything is ASCII, text editing is still a very complex subject once you get into the weeds. You could probably write a book just on handling text overflow

Analemma_ 2y ago

Someday I want to write a lengthy essay on the software-specific variant of the "Seeing Like a State" problem where certain programmers-- angry that the Real World is actually quite complicated-- demand that the world be made simpler to accommodate their wishes for an easier job and cleaner code architecture. This comment is a great example of this phenomenon, as are stupid proposals like "abolish time zones and make everyone live on UTC because I don't like writing time-and-date code".

dsego 2y ago
But why not make the world simpler? Eg. let's simplify those writing systems that are causing issues. Not everything needs to be encoded as text strings. Good thing you mention UTC, because guess what, time zones are a simplification, real local time is on a continuous spectrum.
- Analemma_ 2y ago
  Because that's authoritarian and ghoulish? The thought of forcibly changing (or eliminating) a writing system that may have been in use by millions of people for centuries with no issues, to satisfy the demands of a bunch of web developers who don't want to add some more if-else branches to their text rendering code, gives me the heebie-jeebies. And it wouldn't even save most programmers any effort: for 99.9% of us, this stuff is all abstracted away by libraries anyway. I suspect you might be over-indexing on the specific example of emojis and skin tone modifiers that the article used, but on the Unicode implementation level that's just the same modifier characters that several real languages need to use to be encoded properly. It's not a useless frivolity.
  - dsego 2y ago
    Not sure why writing systems couldn't be improved and simplified for the digital age. It would reduce software complexity and be more efficient and economical.
    - MonkeyClub 2y ago