5 ms·

You can just do `#![forbid(non_ascii_idents)]` in Rust. It'll prevent this kind of attacks completely and you shouldn't need non-ASCII idents anyway.

Tepix 5y ago
That seems like throwing out the baby with the bathwater. We don't all want to go back to the IT stone age of 1963.
- lifthrasiir 5y ago
  `#![forbid(...)]` is a crate-wide attribute, so it is more like a policy (that is good to have if your code would be entirely ASCII).
- Ygg2 5y ago
  Agreed. I use Unicode identifiers to spot shitcode. This would really hamper my detection abilities. Jokes aside, if you're writing Unicode identifiers it means you're not writing your code to be read by a broad audience.
- goldsteinq 5y ago
  You still can use Unicode in comments and string literals. You just can't use non-ASCII characters in identifiers. Unicode in identifiers is just a bad idea. 1. It creates a security consideration with confusable identifiers (and lints don't always catch these) 2. It breaks tooling with RTL identifiers 3. It may not render correctly depending on fonts 4. It may be hard to type depending on keyboard layout 5. There really isn't a good reason to use non-ASCII idents anyway
  - lifthrasiir 5y ago
    > 1. It creates a security consideration with confusable identifiers (and lints don't always catch these) O/0 and I/1/l are confusable characters within ASCII. I'm not kidding here, they are actual entries in the Unicode confusables database [1]. But no one wants to remove those characters from identifiers. [1] For example, https://util.unicode.org/UnicodeJsps/confusables.jsp?a=0&r=None https://util.unicode.org/UnicodeJsps/confusables.jsp?a=0&r=N... > 2. It breaks tooling with RTL identifiers It rather unbreaks tooling with no RTL support. > 3. It may not render correctly depending on fonts So does Unicode in comments and string literals. In fact the purported Trojan "attack" was mostly about string literals. So why should they be allowed in strings but disallowed in identifiers? > 4. It may be hard to type depending on keyboard layout Did you know that not every Latin keyboard layout supports a backquote (`)? This was the actual reason that the repr(expr) shortcut got removed from Python 3 [2]. [2] https://mail.python.org/pipermail/python-ideas/2007-January/000054.html https://mail.python.org/pipermail/python-ideas/2007-January/... > 5. There really isn't a good reason to use non-ASCII idents anyway My canonical answer from the experience is that not every programmer who can understand English documentations can easily write and comprehend English in general. For those people having a non-ASCII identifier support is a great relief, as it frees them from choosing "correct" English identifiers. You can disallow them for your project if you want (or conversely, make it an optional feature disabled by default), but they are relevant for someone else.
    - mkl 5y ago
      Non-ASCII identifiers can be useful for maths too. E.g. I use λ sometimes, especially in Python where "lambda" is a keyword. (I have AutoHotKey and Espanso hotstrings to make typing such symbols easy.)
    - cedilla 5y ago
      > it frees them from choosing "correct" English identifiers Even if you have fluent English skills, sometimes translations just confuse the issue. It's sometimes better to use an untranslated word instead of introducing ambiguity, especially when a term originates from a local law.
      - lifthrasiir 5y ago
        Like CNLabelContactRelationYoungerCousinMothersSiblingsDaughterOrFathersSistersDaughter [1]? :-) You are very much correct. [1] https://news.ycombinator.com/item?id=28712667 https://news.ycombinator.com/item?id=28712667
    - __s 5y ago
      > O/0 and I/1/l are confusable characters within ASCII Which is why the first thing I make sure of when looking at programming fonts is how well they differentiate these characters
      - koheripbal 5y ago
        Agreed - this is literally the first thing I check when selecting the editor's font
    - josephcsible 5y ago
      > O/0 and I/1/l are confusable characters within ASCII. You're mixing up two different ways that people use the word "confusable": things that look similar in some fonts, versus things that look exactly the same regardless of font. I want the latter to be banned from source files but not the former.
  - Tepix 5y ago
    It allows people to code who may not even speak english. That seems more important than your reasons.
    - cema 5y ago
      This argument seems to assume that people who do not know English cannot figure out how to use ASCII character set for identifiers. If so, it is rather incorrect. Is there something else here, maybe?
      - Tepix 5y ago
        Yes if they don't speak english they will want to use non ASCII identifiers because that's what non english words use.
- CodesInChaos 5y ago
  The compiler disallowing them globally might count as that. But individual crates enforcing an "ascii only" policy makes sense, if they never plan to use non-ascii. Personally I'd prefer even one step further: the compiler would disallow them by default, and you can opt into specific character sets/languages at a crate level. e.g. `AllowSpecialCharacters("de")` to enable on special characters common in German.
matheusmoreira 5y ago
> you shouldn't need non-ASCII idents anyway Yes, we do. People from all over the world write software too. They should be able to use the words they know in code. Also, it's totally cool to have mathematical symbols in code. λ, for example. Much more readable than the word lambda. The only reason these symbols are hard to type is our keyboards suck. They can be made easy to type with editor support though.
- jimmaswell 5y ago
  ASCII is the standard for code for good reason. Everyone can type it. Put whatever you want in comments, but you shouldn't make people have to copy/paste your variable names.
  - matheusmoreira 5y ago
    > you shouldn't make people have to copy/paste your variable names The people working on a non-english codebase don't have to. Their keyboards have the symbols they're typing.
    - jimmaswell 5y ago
      You're assuming no international collaboration.
- antris 5y ago
  >Yes, we do. People from all over the world write software too. They should be able to use the words they know in code My native language has non-ASCII characters and I do not expect nor do I want to be able to type them outside string literals. Specifically for the reasons stated in the blog post, among others. Writing in my native language is far, far down in the list of priorities as a professional coder, when security / compatibility are there too. Suggesting that non-native English speakers have to be able to code in their native language also would suggest that non-native coders do not take security / compatibility seriously, which would mean that they are unprofessional. I'm pretty sure that it's not your intention to suggest that, but that's kind of how it comes across. With all the problems eliminated by the use of English and ASCII, it would strike me as amateurish to not use English and ASCII wherever possible.
UncleMeat 5y ago
> It'll prevent this kind of attacks completely It won't. The same approach works just fine in your build specification or other config files. And it doesn't solve the root of this problem, which is that you are compiling source code you don't control and don't audit closely into your binary. Sneaky text is not the only way of getting malicious code through code review.

Tepix 5y ago

That seems like throwing out the baby with the bathwater. We don't all want to go back to the IT stone age of 1963.

lifthrasiir 5y ago
`#![forbid(...)]` is a crate-wide attribute, so it is more like a policy (that is good to have if your code would be entirely ASCII).
Ygg2 5y ago
Agreed. I use Unicode identifiers to spot shitcode. This would really hamper my detection abilities. Jokes aside, if you're writing Unicode identifiers it means you're not writing your code to be read by a broad audience.
goldsteinq 5y ago
You still can use Unicode in comments and string literals. You just can't use non-ASCII characters in identifiers. Unicode in identifiers is just a bad idea. 1. It creates a security consideration with confusable identifiers (and lints don't always catch these) 2. It breaks tooling with RTL identifiers 3. It may not render correctly depending on fonts 4. It may be hard to type depending on keyboard layout 5. There really isn't a good reason to use non-ASCII idents anyway
- lifthrasiir 5y ago
  > 1. It creates a security consideration with confusable identifiers (and lints don't always catch these) O/0 and I/1/l are confusable characters within ASCII. I'm not kidding here, they are actual entries in the Unicode confusables database [1]. But no one wants to remove those characters from identifiers. [1] For example, https://util.unicode.org/UnicodeJsps/confusables.jsp?a=0&r=None https://util.unicode.org/UnicodeJsps/confusables.jsp?a=0&r=N... > 2. It breaks tooling with RTL identifiers It rather unbreaks tooling with no RTL support. > 3. It may not render correctly depending on fonts So does Unicode in comments and string literals. In fact the purported Trojan "attack" was mostly about string literals. So why should they be allowed in strings but disallowed in identifiers? > 4. It may be hard to type depending on keyboard layout Did you know that not every Latin keyboard layout supports a backquote (`)? This was the actual reason that the repr(expr) shortcut got removed from Python 3 [2]. [2] https://mail.python.org/pipermail/python-ideas/2007-January/000054.html https://mail.python.org/pipermail/python-ideas/2007-January/... > 5. There really isn't a good reason to use non-ASCII idents anyway My canonical answer from the experience is that not every programmer who can understand English documentations can easily write and comprehend English in general. For those people having a non-ASCII identifier support is a great relief, as it frees them from choosing "correct" English identifiers. You can disallow them for your project if you want (or conversely, make it an optional feature disabled by default), but they are relevant for someone else.
  - mkl 5y ago
    Non-ASCII identifiers can be useful for maths too. E.g. I use λ sometimes, especially in Python where "lambda" is a keyword. (I have AutoHotKey and Espanso hotstrings to make typing such symbols easy.)
  - cedilla 5y ago
    > it frees them from choosing "correct" English identifiers Even if you have fluent English skills, sometimes translations just confuse the issue. It's sometimes better to use an untranslated word instead of introducing ambiguity, especially when a term originates from a local law.
    - lifthrasiir 5y ago
      Like CNLabelContactRelationYoungerCousinMothersSiblingsDaughterOrFathersSistersDaughter [1]? :-) You are very much correct. [1] https://news.ycombinator.com/item?id=28712667 https://news.ycombinator.com/item?id=28712667
  - __s 5y ago
    > O/0 and I/1/l are confusable characters within ASCII Which is why the first thing I make sure of when looking at programming fonts is how well they differentiate these characters
    - koheripbal 5y ago
      Agreed - this is literally the first thing I check when selecting the editor's font
  - josephcsible 5y ago
    > O/0 and I/1/l are confusable characters within ASCII. You're mixing up two different ways that people use the word "confusable": things that look similar in some fonts, versus things that look exactly the same regardless of font. I want the latter to be banned from source files but not the former.
- Tepix 5y ago
  It allows people to code who may not even speak english. That seems more important than your reasons.
  - cema 5y ago
    This argument seems to assume that people who do not know English cannot figure out how to use ASCII character set for identifiers. If so, it is rather incorrect. Is there something else here, maybe?
    - Tepix 5y ago
      Yes if they don't speak english they will want to use non ASCII identifiers because that's what non english words use.
CodesInChaos 5y ago
The compiler disallowing them globally might count as that. But individual crates enforcing an "ascii only" policy makes sense, if they never plan to use non-ascii. Personally I'd prefer even one step further: the compiler would disallow them by default, and you can opt into specific character sets/languages at a crate level. e.g. `AllowSpecialCharacters("de")` to enable on special characters common in German.

matheusmoreira 5y ago

> you shouldn't need non-ASCII idents anyway Yes, we do. People from all over the world write software too. They should be able to use the words they know in code. Also, it's totally cool to have mathematical symbols in code. λ, for example. Much more readable than the word lambda. The only reason these symbols are hard to type is our keyboards suck. They can be made easy to type with editor support though.

jimmaswell 5y ago
ASCII is the standard for code for good reason. Everyone can type it. Put whatever you want in comments, but you shouldn't make people have to copy/paste your variable names.
- matheusmoreira 5y ago
  > you shouldn't make people have to copy/paste your variable names The people working on a non-english codebase don't have to. Their keyboards have the symbols they're typing.
  - jimmaswell 5y ago
    You're assuming no international collaboration.
antris 5y ago
>Yes, we do. People from all over the world write software too. They should be able to use the words they know in code My native language has non-ASCII characters and I do not expect nor do I want to be able to type them outside string literals. Specifically for the reasons stated in the blog post, among others. Writing in my native language is far, far down in the list of priorities as a professional coder, when security / compatibility are there too. Suggesting that non-native English speakers have to be able to code in their native language also would suggest that non-native coders do not take security / compatibility seriously, which would mean that they are unprofessional. I'm pretty sure that it's not your intention to suggest that, but that's kind of how it comes across. With all the problems eliminated by the use of English and ASCII, it would strike me as amateurish to not use English and ASCII wherever possible.

UncleMeat 5y ago

> It'll prevent this kind of attacks completely It won't. The same approach works just fine in your build specification or other config files. And it doesn't solve the root of this problem, which is that you are compiling source code you don't control and don't audit closely into your binary. Sneaky text is not the only way of getting malicious code through code review.