7 ms·

It feels like the whole “everything is Unicode now” movement was misguided to me. Some things are meant to be simple, and Unicode is massively complex. True, t

It feels like the whole “everything is Unicode now” movement was misguided to me. Some things are meant to be simple, and Unicode is massively complex. True, these are rare-ish corner cases, but every programmer knows it’s only ever a matter of time until you hit any possible corner case. I’m not saying, screw people whose languages don’t fit into ASCII (I’m one of them btw), but I really don’t want to know the vagaries of Unicode representation to inspect a file name. Maybe Unicode needs a “safe mode” where only outright characters are allowed? Or maybe the unavoidable complexity is a design flaw, for some uses anyway. I feel like we’re close to someone discovering Unicode is Turing-complete...

deleted 5y ago
[deleted]
dotancohen 5y ago
> Maybe Unicode needs a “safe mode” where only outright characters are allowed? What is safe for me may not be safe for you. I need RLM characters in my terminal emulator [1], you probably don't. And who knows, maybe RLM characters could prove to be an attack vector, by e.g. displaying filenames or other text backwards. Every Unicode character and code point exists because somebody, somewhere, needs it. [1] https://bugs.kde.org/show_bug.cgi?id=450219 https://bugs.kde.org/show_bug.cgi?id=450219
- mtizim 5y ago
  I highly doubt that last statement is true: 𓂺, ꙮ, 𒀱 .
  - dotancohen 5y ago
    Point taken! I rephrase: Every Unicode character and code point exists because somebody, somewhere, thought that somebody would find it useful.
    - thaumasiotes 5y ago
      That isn't true either; many Unicode characters exist solely because they were part of some other encoding standard that Unicode preserved in its entirety. For example, all of ASCII is preserved in that way. That's also why 囍, a character which does not have any linguistic use, is in there - it was included in an earlier Chinese encoding standard. Nobody thinks it was a good idea to separate U+2126 OHM SIGN from U+03A9 GREEK CAPITAL LETTER OMEGA either.
      - wzdd 5y ago
        Is that true? It seems like screen readers would benefit from knowing the difference between Ohms and Omega, particularly if the surrounding text is in Greek.
        projektfu 5y ago
        What is the likelihood that a Greek article about electronics would use the Ohm sign when the Omega is right there on the keyboard?
        dotancohen 5y ago
        English language physics texts routinely use the proper symbols, despite the fact that e.g. V is right there on the keyboard.
        thaumasiotes 5y ago
        Omega is the proper symbol for Ohms. English-language physics texts use V when they're denominating in volts, too.
        klabb3 5y ago
        So there is no "v that signals the author knows some trivia"? Disappointed.
        5y ago
  - WalterGR 5y ago
    Egyptian Hieroglyphics, Cyrillic glyph variant, and Cuneiform respectively. I had to look them up so I might as well save others the time.
  - tragomaskhalos 5y ago
    Egyptologists would disagree re your first there: it is D53 in the Gardiner sign list, used to spell words including 'urinate', 'poison', 'husband' and 'in the presence of' !
  - dtech 5y ago
    You can definitely argue about whether Unicode should support long-dead or undeciphered languages, but those aren't the characters that cause problems in practice, or at least not more than any random emoji would and those have been embraced with fervor.
- kazinator 5y ago
  > Every Unicode character and code point exists because somebody, somewhere, needs it. Not quite. That and because some twits are willing to cave in and add it, because adding crap to Unicode gives them a sense of purpose which makes them blind to the harm they are causing.
ajuc 5y ago
The problem in this case was that we depend on convention (filenames and extensions) for security. Operating system should already know if a file is executable and desktop should be using this information when presenting files instead of conventional names. Yes it would mean that when you download a file you have to set the executable attribute before you can run it. It's already almost exactly like that on Linux. That's a good thing.
- thaumasiotes 5y ago
  > Operating system should already know if a file is executable and desktop should be using this information when presenting files instead of conventional names. That already happens. If you set the "Details" view in Windows, it will display one file per row with various columns showing file metadata, including the file type. So for the example in the post, you'd see "[Word icon] annexe.doc - [the date] - Application" as opposed to "[Word icon] annexe.doc - [the date] - Microsoft Word 97 - 2003 Document".
  - ajuc 5y ago
    I meant more like huge red and green exclamation mark near the file icon, not "type: Application" hidden in rarely shown places. In linux it's pretty obvious: https://www.rootusers.com/wp-content/uploads/2018/01/gnome-terminal.png https://www.rootusers.com/wp-content/uploads/2018/01/gnome-t... The main advantage is that the UI for opening documents (vim file) is different from the UI for running executables (./file) which is different from the UI for running executables as administrator (sudo ./file). This makes it impossible to accidentally execute a file that looks like a document.
- trasz 5y ago
  On Linux you have another problem though: you can have several files with the exact same name, just encoded differently.
- jcelerier 5y ago
  As a desktop user, it makes me physically ill everytime I have to do that. I'd honestly pay for someone to make auto-executing the default on most common Linux distros. Like, in 20+ years of using computers, I don't remember one time where I downloaded something and did not want to run it afterwards.
AnIdiotOnTheNet 5y ago
I think I agree. Trying to encode all of human written language in one standard is just too huge a scope, and then you shove all that complexity into every layer of computing except just being in the kind of documents that need that level of complexity. It's definitely an extreme view[0], and I'm open to being argued out of it, but I really think we should use a different and much simpler encoding for the vast majority of things even if it doesn't perfectly reproduce the written text of the language. Writing systems encoded the language in a way fit for the medium of paper, and maybe we need to consider computers a different medium and adapt our language to them differently too. [0] and probably anglo-centric.

deleted 5y ago

[deleted]

dotancohen 5y ago

> Maybe Unicode needs a “safe mode” where only outright characters are allowed? What is safe for me may not be safe for you. I need RLM characters in my terminal emulator [1], you probably don't. And who knows, maybe RLM characters could prove to be an attack vector, by e.g. displaying filenames or other text backwards. Every Unicode character and code point exists because somebody, somewhere, needs it. [1] https://bugs.kde.org/show_bug.cgi?id=450219 https://bugs.kde.org/show_bug.cgi?id=450219

mtizim 5y ago
I highly doubt that last statement is true: 𓂺, ꙮ, 𒀱 .
- dotancohen 5y ago
  Point taken! I rephrase: Every Unicode character and code point exists because somebody, somewhere, thought that somebody would find it useful.
  - thaumasiotes 5y ago
    That isn't true either; many Unicode characters exist solely because they were part of some other encoding standard that Unicode preserved in its entirety. For example, all of ASCII is preserved in that way. That's also why 囍, a character which does not have any linguistic use, is in there - it was included in an earlier Chinese encoding standard. Nobody thinks it was a good idea to separate U+2126 OHM SIGN from U+03A9 GREEK CAPITAL LETTER OMEGA either.
    - wzdd 5y ago
      Is that true? It seems like screen readers would benefit from knowing the difference between Ohms and Omega, particularly if the surrounding text is in Greek.
      - projektfu 5y ago
        What is the likelihood that a Greek article about electronics would use the Ohm sign when the Omega is right there on the keyboard?
        dotancohen 5y ago
        English language physics texts routinely use the proper symbols, despite the fact that e.g. V is right there on the keyboard.
        thaumasiotes 5y ago
        Omega is the proper symbol for Ohms. English-language physics texts use V when they're denominating in volts, too.
        klabb3 5y ago
        So there is no "v that signals the author knows some trivia"? Disappointed.
      - 5y ago
- WalterGR 5y ago
  Egyptian Hieroglyphics, Cyrillic glyph variant, and Cuneiform respectively. I had to look them up so I might as well save others the time.
- tragomaskhalos 5y ago
  Egyptologists would disagree re your first there: it is D53 in the Gardiner sign list, used to spell words including 'urinate', 'poison', 'husband' and 'in the presence of' !
- dtech 5y ago
  You can definitely argue about whether Unicode should support long-dead or undeciphered languages, but those aren't the characters that cause problems in practice, or at least not more than any random emoji would and those have been embraced with fervor.
kazinator 5y ago
> Every Unicode character and code point exists because somebody, somewhere, needs it. Not quite. That and because some twits are willing to cave in and add it, because adding crap to Unicode gives them a sense of purpose which makes them blind to the harm they are causing.

ajuc 5y ago

The problem in this case was that we depend on convention (filenames and extensions) for security. Operating system should already know if a file is executable and desktop should be using this information when presenting files instead of conventional names. Yes it would mean that when you download a file you have to set the executable attribute before you can run it. It's already almost exactly like that on Linux. That's a good thing.

thaumasiotes 5y ago
> Operating system should already know if a file is executable and desktop should be using this information when presenting files instead of conventional names. That already happens. If you set the "Details" view in Windows, it will display one file per row with various columns showing file metadata, including the file type. So for the example in the post, you'd see "[Word icon] annexe.doc - [the date] - Application" as opposed to "[Word icon] annexe.doc - [the date] - Microsoft Word 97 - 2003 Document".
- ajuc 5y ago
  I meant more like huge red and green exclamation mark near the file icon, not "type: Application" hidden in rarely shown places. In linux it's pretty obvious: https://www.rootusers.com/wp-content/uploads/2018/01/gnome-terminal.png https://www.rootusers.com/wp-content/uploads/2018/01/gnome-t... The main advantage is that the UI for opening documents (vim file) is different from the UI for running executables (./file) which is different from the UI for running executables as administrator (sudo ./file). This makes it impossible to accidentally execute a file that looks like a document.
trasz 5y ago
On Linux you have another problem though: you can have several files with the exact same name, just encoded differently.
jcelerier 5y ago
As a desktop user, it makes me physically ill everytime I have to do that. I'd honestly pay for someone to make auto-executing the default on most common Linux distros. Like, in 20+ years of using computers, I don't remember one time where I downloaded something and did not want to run it afterwards.

AnIdiotOnTheNet 5y ago

I think I agree. Trying to encode all of human written language in one standard is just too huge a scope, and then you shove all that complexity into every layer of computing except just being in the kind of documents that need that level of complexity. It's definitely an extreme view[0], and I'm open to being argued out of it, but I really think we should use a different and much simpler encoding for the vast majority of things even if it doesn't perfectly reproduce the written text of the language. Writing systems encoded the language in a way fit for the medium of paper, and maybe we need to consider computers a different medium and adapt our language to them differently too. [0] and probably anglo-centric.