4 ms·

Ehhhh... Interesting philosophically, and we might see a practical attack maybe eventually, but most source code editors and diff reviewers that I've encountere

by Groxx 5y ago

Ehhhh... Interesting philosophically, and we might see a practical attack maybe eventually, but most source code editors and diff reviewers that I've encountered show all non-printable characters VERY visibly. Because they matter, and always have - "func asdf()" is very different from "func as<zwsp>df()". If I saw a pile of non-printable control characters intermixed in code in a diff, there's absolutely no way I'd allow that merge. IOCCC entries will absolutely become more fun though.

lifthrasiir 5y ago
> IOCCC entries will absolutely become more fun though. IOCCC doesn't allow unescaped octets with high bit set [1], so even that's no go. [1] https://www.ioccc.org/2020/rules.txt https://www.ioccc.org/2020/rules.txt (rule 13)
- Groxx 5y ago
  Aww. But also of course they've already addressed this.
- saagarjha 5y ago
  I am very curious which program abused this and forced the creation of that rule.
  - lifthrasiir 5y ago
    Probably 2000/briddlebane [1]. But it is more like a guard against compatibility issues. [1] https://www.ioccc.org/2000/briddlebane.c https://www.ioccc.org/2000/briddlebane.c vs. https://www.ioccc.org/2000/briddlebane.orig.c https://www.ioccc.org/2000/briddlebane.orig.c
- GlitchMr 5y ago
  Well, technically the rule only talks about entries that "fail to compile". An entry that still compiles is fine, see rule 12. In practice this means the Unicode abuse like this is only allowed in strings.
  - lifthrasiir 5y ago
    When the rule was originally introduced in 2001 [1] it was a total ban. It seems that the rule was slightly relaxed in 2013 [2], but I think it still massively discourages any octet >= 128 because there is no portable way to set the input encoding (like GCC `-finput-charset`, which is ignored by Clang AFAIK). [1] https://www.ioccc.org/2001/rules https://www.ioccc.org/2001/rules [2] https://www.ioccc.org/2013/rules.txt https://www.ioccc.org/2013/rules.txt
deleted 5y ago
[deleted]
Jach 5y ago
I wouldn't be so sure about visibility since it seems most code editors and programming languages want to support more unicode, not less... One of my hobbies used to be annually running a regex search through the company's millions of lines of java to see how much of an increase there was in non-printable spaces (0x200b) in java method names or other symbols. Eclipse at least wouldn't show them by default, I don't remember IntelliJ's behavior, but most people wouldn't know they were there. I was aware of only one time when it impacted someone who typed in a whole identifier by sight but the reference included a 200b and they were stuck for a bit figuring out why things didn't work. But I agree the trick (hard to call it an attack or even bug) is fun, in the same way as the earlier tricks of fake filename extensions. And terribly obvious, even with the limitations of default code viewers, and with no plausible deniability once caught, so it's pretty overblown for practical considerations. The intentionally introduced Linux kernel bugs from several months ago were far more significant a lesson for people to learn from, and they didn't rely on any unicode tricks but on much simpler tricks that were also somewhat plausibly deniable to chalk up to an oopsie.
- Groxx 5y ago
  yeah, I've had an identifier or two like that in Ruby in the past :) always worth a few facepalm-riddled lols when sharing the final result with the rest of the team, especially since it often meant they copied the func from Stack Overflow or some equivalent. Most of what I've encountered though has been due to a lack of unicode support, and related growing pains in adopting full UTF-8. E.g. much of the Eclipse issues I saw were due to UTF-16 weirdness and stuff encoded in ShiftJIS or whatever flavor of Windows encoding you used, and all those garbled files due to missing magic-encoding-bytes in files. UTF-8 support "completing" in tools largely cleaned all that up, since they detected the encoding, converted to UTF-8, and showed abnormal stuff as the abnormalities they were all along. I mean, that's probably because taking a deep look at supporting UTF-8 meant taking a deep look at many of their latent text bugs and finally fixing them, but it still happened around the same time, and "X editor now supports UTF-8" also marked a dramatic increase in "... and now shows <nbsp> explicitly!" and similar things.

by Groxx 5y ago

lifthrasiir 5y ago

> IOCCC entries will absolutely become more fun though. IOCCC doesn't allow unescaped octets with high bit set [1], so even that's no go. [1] https://www.ioccc.org/2020/rules.txt https://www.ioccc.org/2020/rules.txt (rule 13)

Groxx 5y ago
Aww. But also of course they've already addressed this.
saagarjha 5y ago
I am very curious which program abused this and forced the creation of that rule.
- lifthrasiir 5y ago
  Probably 2000/briddlebane [1]. But it is more like a guard against compatibility issues. [1] https://www.ioccc.org/2000/briddlebane.c https://www.ioccc.org/2000/briddlebane.c vs. https://www.ioccc.org/2000/briddlebane.orig.c https://www.ioccc.org/2000/briddlebane.orig.c
GlitchMr 5y ago
Well, technically the rule only talks about entries that "fail to compile". An entry that still compiles is fine, see rule 12. In practice this means the Unicode abuse like this is only allowed in strings.
- lifthrasiir 5y ago
  When the rule was originally introduced in 2001 [1] it was a total ban. It seems that the rule was slightly relaxed in 2013 [2], but I think it still massively discourages any octet >= 128 because there is no portable way to set the input encoding (like GCC `-finput-charset`, which is ignored by Clang AFAIK). [1] https://www.ioccc.org/2001/rules https://www.ioccc.org/2001/rules [2] https://www.ioccc.org/2013/rules.txt https://www.ioccc.org/2013/rules.txt

deleted 5y ago

[deleted]

Jach 5y ago

I wouldn't be so sure about visibility since it seems most code editors and programming languages want to support more unicode, not less... One of my hobbies used to be annually running a regex search through the company's millions of lines of java to see how much of an increase there was in non-printable spaces (0x200b) in java method names or other symbols. Eclipse at least wouldn't show them by default, I don't remember IntelliJ's behavior, but most people wouldn't know they were there. I was aware of only one time when it impacted someone who typed in a whole identifier by sight but the reference included a 200b and they were stuck for a bit figuring out why things didn't work. But I agree the trick (hard to call it an attack or even bug) is fun, in the same way as the earlier tricks of fake filename extensions. And terribly obvious, even with the limitations of default code viewers, and with no plausible deniability once caught, so it's pretty overblown for practical considerations. The intentionally introduced Linux kernel bugs from several months ago were far more significant a lesson for people to learn from, and they didn't rely on any unicode tricks but on much simpler tricks that were also somewhat plausibly deniable to chalk up to an oopsie.

Groxx 5y ago
yeah, I've had an identifier or two like that in Ruby in the past :) always worth a few facepalm-riddled lols when sharing the final result with the rest of the team, especially since it often meant they copied the func from Stack Overflow or some equivalent. Most of what I've encountered though has been due to a lack of unicode support, and related growing pains in adopting full UTF-8. E.g. much of the Eclipse issues I saw were due to UTF-16 weirdness and stuff encoded in ShiftJIS or whatever flavor of Windows encoding you used, and all those garbled files due to missing magic-encoding-bytes in files. UTF-8 support "completing" in tools largely cleaned all that up, since they detected the encoding, converted to UTF-8, and showed abnormal stuff as the abnormalities they were all along. I mean, that's probably because taking a deep look at supporting UTF-8 meant taking a deep look at many of their latent text bugs and finally fixing them, but it still happened around the same time, and "X editor now supports UTF-8" also marked a dramatic increase in "... and now shows <nbsp> explicitly!" and similar things.