3 ms·

https://github.com/rust-lang/rust/issues/28979 https://github.com/rust-lang/rust/issues/28979 plenty of discussion here on Unicode including homoglyph attacks.

by ximeng 5y ago

https://github.com/rust-lang/rust/issues/28979 https://github.com/rust-lang/rust/issues/28979 plenty of discussion here on Unicode including homoglyph attacks. This is for Rust but has links to Go and Zig. The Unicode standard also has extensive discussion, for example https://unicode.org/reports/tr31/ https://unicode.org/reports/tr31/ and http://unicode.org/reports/tr39/ http://unicode.org/reports/tr39/ on identifiers and security. In general a multilayer solution is needed: compilers, linters, Unicode standard, merge tools, editors, and so on.

rurban 5y ago
But they still don't get it right, they explicitly allow not identifiable Unicode identifiers. The C20 committee recently allowed also insecure identifiers, completely ignoring the Unicode identifier guidelines. They stated that nobody cares, everybody wants them and making them secure would need the entire Unicode database. Why do they allow noobs into such committees? What is needed are the normalization tables (tiny), the script list (tiny) and the two xid lists.
- estebank 5y ago
  > they explicitly allow not identifiable Unicode identifiers. [...] They stated that nobody cares, everybody wants them and making them secure would need the entire Unicode database. Could you elaborate? rustc ships with the entire Unicode db and only allows indents with codepoints advertised by Unicode as allowed in indents. The closest to walking off the beaten path is a (still unmerged) parser recovery PR that accepts emojis as identifiers if and only if a parse error would otherwise occur as a way to avoid knock down errors when someone tries to use them.
  - rurban 5y ago
    For identifier security you don't need the entire Unicode DB. Only rust or glibc would do that, nobody else. You need the XID_Start/Continue list of bits, a single normalization table if NFC (or two if NFD), the scripts list (ranges of a single byte), and a bit of logic. With confusables I'm not sure. That's about 2k vs 20m.

by ximeng 5y ago

rurban 5y ago

But they still don't get it right, they explicitly allow not identifiable Unicode identifiers. The C20 committee recently allowed also insecure identifiers, completely ignoring the Unicode identifier guidelines. They stated that nobody cares, everybody wants them and making them secure would need the entire Unicode database. Why do they allow noobs into such committees? What is needed are the normalization tables (tiny), the script list (tiny) and the two xid lists.

estebank 5y ago
> they explicitly allow not identifiable Unicode identifiers. [...] They stated that nobody cares, everybody wants them and making them secure would need the entire Unicode database. Could you elaborate? rustc ships with the entire Unicode db and only allows indents with codepoints advertised by Unicode as allowed in indents. The closest to walking off the beaten path is a (still unmerged) parser recovery PR that accepts emojis as identifiers if and only if a parse error would otherwise occur as a way to avoid knock down errors when someone tries to use them.
- rurban 5y ago
  For identifier security you don't need the entire Unicode DB. Only rust or glibc would do that, nobody else. You need the XID_Start/Continue list of bits, a single normalization table if NFC (or two if NFD), the scripts list (ranges of a single byte), and a bit of logic. With confusables I'm not sure. That's about 2k vs 20m.