5 ms·

Are there any lists of unicode characters (like the OWASP one) that should be blacklisted from most apps (not just for XSS, but even for desktop apps)? Are the

Are there any lists of unicode characters (like the OWASP one) that should be blacklisted from most apps (not just for XSS, but even for desktop apps)? Are there any good security guides/best practices for unicode sanitation?

harambae 5y ago
Not a full security guide, but if you haven't seen this before it's useful to have... https://github.com/danielmiessler/SecLists/blob/master/Fuzzing/big-list-of-naughty-strings.txt https://github.com/danielmiessler/SecLists/blob/master/Fuzzi...
- adamrezich 5y ago
  I've seen this before but either this is new since last time or I missed it, either way: lol # Human injection # # Strings which may cause human to reinterpret worldview If you're reading this, you've been in a coma for almost 20 years now. We're trying a new technique. We don't know where this message will end up in your dream, but we hope it works. Please wake up, we miss you.
  - p_j_w 5y ago
    I'm not falling for this one, I know no one misses me!
wongarsu 5y ago
How are users supposed to write "עבור אל duckduckgo.com כדי לחפש באינטרנט" without \u202E? It's perfectly normal for RTL languages to switch text direction in the middle of a sentence.
- OJFord 5y ago
  Do you read left RTL, middle LTR, right RTL; or right RTL, middle LTR, left RTL? (Just curious.)
  - anamexis 5y ago
    Imagine it was the other way around: like you wanted to reference תֵּל־אָבִיב-יָפוֹ in the middle of an English sentence. That, but reversed.
    - OJFord 5y ago
      It is different though, because with LTR RTL LTR: 1) once you've read the first LTR you end in the right place for the RTL 2) even if you consider the LTRs to be two distinct documents it doesn't change the order. I can imagine the LTR 'breaking' the RTL text such that the sections are treated separately. But I understand what you were saying, so thanks for the answer!
      - anamexis 5y ago
        On 1, no you’re not. After the first LTR you’re at the end of the RTL. On 2, I’m not sure I understand. The situation is the same for both primary directions.
  - kingcharles 5y ago
    > Do you read left RTL, middle LTR, right RTL; or right RTL, middle LTR, left RTL? (Just curious.) YES.
- raphlinus 5y ago
  That should just render correctly thanks to the BiDi algorithm. The "override" control characters are a heavy hammer, and are extremely rarely needed. In fact, at this point I think it's likely that malicious use of these code points significantly outweighs correct use. There are legitimate uses of BiDi control characters. My favorite one from my time on Android was the string "Google+", which would render as "+Google" in an RTL paragraph. The translators would usually "fix" this by just flipping the string so that it was "+Google", which would render correctly, but be incorrect when cut'n'pasted, read by a screen reader, etc. The correct solution is to use a left-to-right mark. The string "Google\u{200e}+" renders correctly in both LTR and RTL flow. And these "mark" characters are basically harmless, they cannot profoundly change the order, they just fix some of these ambiguous cases. Correct use of BiDi control characters is explained here: https://www.w3.org/International/questions/qa-bidi-unicode-controls https://www.w3.org/International/questions/qa-bidi-unicode-c...
- kingcharles 5y ago
  And then you get Arabic and English text quoted in Japanese vertical RTL text and that's the story of how I actually died.
sp332 5y ago
I don't think this is a good place for a blacklist. Text effects should be encapsulated and reset at the end of the text block, the way bold or italic effects are.
sterlind 5y ago
please don't blacklist U+202D and U+202E or the Private Use Area. my conlang has a right-to-left cursive script, and it's not in Unicode. the characters live in the PUA and my font renders them as a fallback. there's no mechanism for fonts to ask for RTL, so I have to use bidi override.
- bawolff 5y ago
  I do think its kind of sad that the PUA doesnt have various areas with different properties (RTL, whitespace, joining, etc)
bawolff 5y ago
Put it inside a <span dir="auto"> ? Anyways unicide category Cf is probably what you are looking for, but blocking them is probably wrong as they serve an important function.
missblit 5y ago
For unicode security considerations see http://www.unicode.org/reports/tr36/ http://www.unicode.org/reports/tr36/ The report is divided into visual and non-visual security issues. Our old friend RTL override is covered, but mostly in the context of URLs.

harambae 5y ago

Not a full security guide, but if you haven't seen this before it's useful to have... https://github.com/danielmiessler/SecLists/blob/master/Fuzzing/big-list-of-naughty-strings.txt https://github.com/danielmiessler/SecLists/blob/master/Fuzzi...

adamrezich 5y ago
I've seen this before but either this is new since last time or I missed it, either way: lol # Human injection # # Strings which may cause human to reinterpret worldview If you're reading this, you've been in a coma for almost 20 years now. We're trying a new technique. We don't know where this message will end up in your dream, but we hope it works. Please wake up, we miss you.
- p_j_w 5y ago
  I'm not falling for this one, I know no one misses me!

wongarsu 5y ago

How are users supposed to write "עבור אל duckduckgo.com כדי לחפש באינטרנט" without \u202E? It's perfectly normal for RTL languages to switch text direction in the middle of a sentence.

OJFord 5y ago
Do you read left RTL, middle LTR, right RTL; or right RTL, middle LTR, left RTL? (Just curious.)
- anamexis 5y ago
  Imagine it was the other way around: like you wanted to reference תֵּל־אָבִיב-יָפוֹ in the middle of an English sentence. That, but reversed.
  - OJFord 5y ago
    It is different though, because with LTR RTL LTR: 1) once you've read the first LTR you end in the right place for the RTL 2) even if you consider the LTRs to be two distinct documents it doesn't change the order. I can imagine the LTR 'breaking' the RTL text such that the sections are treated separately. But I understand what you were saying, so thanks for the answer!
    - anamexis 5y ago
      On 1, no you’re not. After the first LTR you’re at the end of the RTL. On 2, I’m not sure I understand. The situation is the same for both primary directions.
- kingcharles 5y ago
  > Do you read left RTL, middle LTR, right RTL; or right RTL, middle LTR, left RTL? (Just curious.) YES.
raphlinus 5y ago
That should just render correctly thanks to the BiDi algorithm. The "override" control characters are a heavy hammer, and are extremely rarely needed. In fact, at this point I think it's likely that malicious use of these code points significantly outweighs correct use. There are legitimate uses of BiDi control characters. My favorite one from my time on Android was the string "Google+", which would render as "+Google" in an RTL paragraph. The translators would usually "fix" this by just flipping the string so that it was "+Google", which would render correctly, but be incorrect when cut'n'pasted, read by a screen reader, etc. The correct solution is to use a left-to-right mark. The string "Google\u{200e}+" renders correctly in both LTR and RTL flow. And these "mark" characters are basically harmless, they cannot profoundly change the order, they just fix some of these ambiguous cases. Correct use of BiDi control characters is explained here: https://www.w3.org/International/questions/qa-bidi-unicode-controls https://www.w3.org/International/questions/qa-bidi-unicode-c...
kingcharles 5y ago
And then you get Arabic and English text quoted in Japanese vertical RTL text and that's the story of how I actually died.

sp332 5y ago

I don't think this is a good place for a blacklist. Text effects should be encapsulated and reset at the end of the text block, the way bold or italic effects are.

sterlind 5y ago

please don't blacklist U+202D and U+202E or the Private Use Area. my conlang has a right-to-left cursive script, and it's not in Unicode. the characters live in the PUA and my font renders them as a fallback. there's no mechanism for fonts to ask for RTL, so I have to use bidi override.

bawolff 5y ago
I do think its kind of sad that the PUA doesnt have various areas with different properties (RTL, whitespace, joining, etc)

bawolff 5y ago

Put it inside a <span dir="auto"> ? Anyways unicide category Cf is probably what you are looking for, but blocking them is probably wrong as they serve an important function.

missblit 5y ago

For unicode security considerations see http://www.unicode.org/reports/tr36/ http://www.unicode.org/reports/tr36/ The report is divided into visual and non-visual security issues. Our old friend RTL override is covered, but mostly in the context of URLs.