4 ms·

Can you point me to good resources to learn more about Unicode normalization especially for passwords? What are the gotchas? One the one hand, when users are al

Can you point me to good resources to learn more about Unicode normalization especially for passwords? What are the gotchas? One the one hand, when users are allowed to enter any character, they often have difficulties to log in when they are aboard and not use their own devices. Though, I don't know if every Russian (for instance) would be able to enter a "English" password.

lrem 9y ago
I believe, as a matter of necessity, every PC out there will allow any user to enter ASCII, even if it's not within their contry language's alphabet.
ademarre 9y ago
https://tools.ietf.org/html/rfc8265 https://tools.ietf.org/html/rfc8265
drdaeman 9y ago
> Though, I don't know if every Russian (for instance) would be able to enter a "English" password. I believe in every country with non-Latin alphabet users have two keyboard layouts: local (e.g. Russian) and English/Latin. All major OSes (desktop, tablet or mobile) have install-time defaults like this, and I never ever saw any divergence from this rule, except for removing native layout if it's deemed unnecessary and going English-only, or configuring system to use some third language (e.g. a foreigner setting up a device purchased abroad, changing the defaults). I mean, I never saw anyone in their sane mind removing English layout and keeping only the local. It is virtually impossible to use computers without having to type in some Latin characters on some occasions, like typing URLs or email addresses. Therefore, I think you may reasonable safely assume capability to enter basic Latin. However, you should also consider that user may forget to switch the keyboard layout (happens all the time), so if you want to do some password transformations like case-insensitive password matching (due to Caps Lock), and you have significant foreign user base, you may also want to consider common keyboard layouts issues (e.g. "hunter2" <-> "ргтеук2" for QWERTY <-> Russian).
- spiffytech 9y ago
  Case-insensitive password comparison in a very bad idea because it strips a lot of entropy from the password pool.
  - drdaeman 9y ago
    My bad, I meant not really case-insensitive but flipped case, to account for possible Caps Lock state. I.e. for "Hunter2" you also accept "hUNTER2" (but "HuNTeR2" is still not valid). Facebook does this. This is only a single bit compromise, and if there are a lot of users affected, benefits may overweigh the slight security downgrade.
cakoose 9y ago
In short, there are multiple ways to represent the same character in Unicode: https://stackoverflow.com/questions/16173328 https://stackoverflow.com/questions/16173328 @ademarre's RFC link looks like a good place to look for guidance.
rdslw 9y ago
This is very good primer on the topic, including unicode normalization, punycode and others, with ultra short python examples and links to more info. https://www.b-list.org/weblog/2018/feb/11/usernames/ https://www.b-list.org/weblog/2018/feb/11/usernames/

lrem 9y ago

I believe, as a matter of necessity, every PC out there will allow any user to enter ASCII, even if it's not within their contry language's alphabet.

ademarre 9y ago

https://tools.ietf.org/html/rfc8265 https://tools.ietf.org/html/rfc8265

drdaeman 9y ago

> Though, I don't know if every Russian (for instance) would be able to enter a "English" password. I believe in every country with non-Latin alphabet users have two keyboard layouts: local (e.g. Russian) and English/Latin. All major OSes (desktop, tablet or mobile) have install-time defaults like this, and I never ever saw any divergence from this rule, except for removing native layout if it's deemed unnecessary and going English-only, or configuring system to use some third language (e.g. a foreigner setting up a device purchased abroad, changing the defaults). I mean, I never saw anyone in their sane mind removing English layout and keeping only the local. It is virtually impossible to use computers without having to type in some Latin characters on some occasions, like typing URLs or email addresses. Therefore, I think you may reasonable safely assume capability to enter basic Latin. However, you should also consider that user may forget to switch the keyboard layout (happens all the time), so if you want to do some password transformations like case-insensitive password matching (due to Caps Lock), and you have significant foreign user base, you may also want to consider common keyboard layouts issues (e.g. "hunter2" <-> "ргтеук2" for QWERTY <-> Russian).

spiffytech 9y ago
Case-insensitive password comparison in a very bad idea because it strips a lot of entropy from the password pool.
- drdaeman 9y ago
  My bad, I meant not really case-insensitive but flipped case, to account for possible Caps Lock state. I.e. for "Hunter2" you also accept "hUNTER2" (but "HuNTeR2" is still not valid). Facebook does this. This is only a single bit compromise, and if there are a lot of users affected, benefits may overweigh the slight security downgrade.

cakoose 9y ago

In short, there are multiple ways to represent the same character in Unicode: https://stackoverflow.com/questions/16173328 https://stackoverflow.com/questions/16173328 @ademarre's RFC link looks like a good place to look for guidance.

rdslw 9y ago

This is very good primer on the topic, including unicode normalization, punycode and others, with ultra short python examples and links to more info. https://www.b-list.org/weblog/2018/feb/11/usernames/ https://www.b-list.org/weblog/2018/feb/11/usernames/