4 ms·

Usernames should be fully case-insensitive. It's trivial to store usernames and email addresses in all lowercase and transform any input to lowercase before com

Usernames should be fully case-insensitive. It's trivial to store usernames and email addresses in all lowercase and transform any input to lowercase before comparing. Case-insensitive comparison is not the same as convert-to-lowercase-and-compare. You should also normalize the input or you're going to set up issues where an account created on one device with one locale will be impossible to log into on another device with another locale.

myroon5 9y ago
Interested in this one. Could you provide a specific example?
- gbacon 9y ago
  Say a German user’s chosen name is spaß (“fun”). In all uppercase, that name is equivalent to SPASS or perhaps SPASZ and as of last year[0] SPAẞ. [0]: https://en.wikipedia.org/wiki/Capital_ẞ https://en.wikipedia.org/wiki/Capital_ẞ
  - eCa 9y ago
    I have never seen ”ß” meaning ”sz” (or anything other than ”ss”), can you point me to a source?
    - jwilk 9y ago
      From the Wikipedia article: Designs for an SZ ligature has been part of typesets from the 1910s until the 1950s, but in the opinion of Herrmann (2011), the long-standing alphabetic equivalence of ß with ss and the abolition of the optional capitalization of ß as SZ (abandoned in 1996⁽¹¹⁾) means that this is no longer a valid option that "Germans would want to use on a regular basis". ¹¹ until 1996, it was recommended to render ß as SS in allcaps except when there was ambiguity, in which case it should be rendered as SZ. The common example for such a case was IN MASZEN (in Maßen "in moderate amounts") vs. IN MASSEN (in Massen "in massive amounts"), where the difference between the spelling in ß vs. ss could actually reverse the conveyed meaning. Also, https://en.wikipedia.org/wiki/%C3%9F https://en.wikipedia.org/wiki/%C3%9F contains a concrete example: https://en.wikipedia.org/wiki/File:ABSCHUSZGERAET.jpg https://en.wikipedia.org/wiki/File:ABSCHUSZGERAET.jpg
- torstenvl 9y ago
  For the first, any situation where there isn't a 1:1 coupling of capital and lowercase letters would result in collision or ambiguity upon a .tolower() For the second, one OS, locale, and input method may result in Unicode fully-composed forms, and another in fully-decomposed forms. Is ø one code point or two? The answer is: it depends on your software environment. If you don't normalize your input, you could have a situation where an account created on, say, Win XP, might be inaccessible for the average user from iOS. There's actually a pretty good StackOverflow post on dealing with this issue in Python: https://stackoverflow.com/questions/319426/how-do-i-do-a-case-insensitive-string-comparison https://stackoverflow.com/questions/319426/how-do-i-do-a-cas...
  - dmichulke 9y ago
    > If you don't normalize your input, you could have a situation where an account created on, say, Win XP, might be inaccessible for the average user from iOS. This only applies if you handle the normalization in the front-end. But you should normalize it in the back-end, right? Or am I missing something here? Edit: Answered further down by wongarsu
    - torstenvl 9y ago
      Yeah my bad for the ambiguity. I meant normalize the input into the database, i.e., into the backend.
- wongarsu 9y ago
  English has the letter “I“ with lowercase form “i“. Turkish has the letter “I“ with lowercase form “ı“ and the letter “İ“ with lowercase form “i“. This means for any word containing the letter “i“ case conversion rules differ between Englisch and Turkish. That's why modern string APIs ask you which locale to use for case conversion.
bzbarsky 9y ago
This part of the article is just so English-specific it hurts. It's assuming that usernames will all be ASCII, basically. There is nothing trivial about either "lowercasing" or "case-insensitive compares" in general Unicode strings...
- eCa 9y ago
  In Perl5 there’s fc() which handles that. https://perldoc.perl.org/functions/fc.html https://perldoc.perl.org/functions/fc.html
  - bzbarsky 9y ago
    "handles" in what sense? It needs to give different answers for the same letter depending on whether the text is English or Turkish, for example, so I don't see how it can "handle" it in general. Reading the documentation you link to, the answer in the fine print is that fc() gives you incorrect results for Turkish text.

myroon5 9y ago

Interested in this one. Could you provide a specific example?

gbacon 9y ago
Say a German user’s chosen name is spaß (“fun”). In all uppercase, that name is equivalent to SPASS or perhaps SPASZ and as of last year[0] SPAẞ. [0]: https://en.wikipedia.org/wiki/Capital_ẞ https://en.wikipedia.org/wiki/Capital_ẞ
- eCa 9y ago
  I have never seen ”ß” meaning ”sz” (or anything other than ”ss”), can you point me to a source?
  - jwilk 9y ago
    From the Wikipedia article: Designs for an SZ ligature has been part of typesets from the 1910s until the 1950s, but in the opinion of Herrmann (2011), the long-standing alphabetic equivalence of ß with ss and the abolition of the optional capitalization of ß as SZ (abandoned in 1996⁽¹¹⁾) means that this is no longer a valid option that "Germans would want to use on a regular basis". ¹¹ until 1996, it was recommended to render ß as SS in allcaps except when there was ambiguity, in which case it should be rendered as SZ. The common example for such a case was IN MASZEN (in Maßen "in moderate amounts") vs. IN MASSEN (in Massen "in massive amounts"), where the difference between the spelling in ß vs. ss could actually reverse the conveyed meaning. Also, https://en.wikipedia.org/wiki/%C3%9F https://en.wikipedia.org/wiki/%C3%9F contains a concrete example: https://en.wikipedia.org/wiki/File:ABSCHUSZGERAET.jpg https://en.wikipedia.org/wiki/File:ABSCHUSZGERAET.jpg
torstenvl 9y ago
For the first, any situation where there isn't a 1:1 coupling of capital and lowercase letters would result in collision or ambiguity upon a .tolower() For the second, one OS, locale, and input method may result in Unicode fully-composed forms, and another in fully-decomposed forms. Is ø one code point or two? The answer is: it depends on your software environment. If you don't normalize your input, you could have a situation where an account created on, say, Win XP, might be inaccessible for the average user from iOS. There's actually a pretty good StackOverflow post on dealing with this issue in Python: https://stackoverflow.com/questions/319426/how-do-i-do-a-case-insensitive-string-comparison https://stackoverflow.com/questions/319426/how-do-i-do-a-cas...
- dmichulke 9y ago
  > If you don't normalize your input, you could have a situation where an account created on, say, Win XP, might be inaccessible for the average user from iOS. This only applies if you handle the normalization in the front-end. But you should normalize it in the back-end, right? Or am I missing something here? Edit: Answered further down by wongarsu
  - torstenvl 9y ago
    Yeah my bad for the ambiguity. I meant normalize the input into the database, i.e., into the backend.
wongarsu 9y ago
English has the letter “I“ with lowercase form “i“. Turkish has the letter “I“ with lowercase form “ı“ and the letter “İ“ with lowercase form “i“. This means for any word containing the letter “i“ case conversion rules differ between Englisch and Turkish. That's why modern string APIs ask you which locale to use for case conversion.

bzbarsky 9y ago

This part of the article is just so English-specific it hurts. It's assuming that usernames will all be ASCII, basically. There is nothing trivial about either "lowercasing" or "case-insensitive compares" in general Unicode strings...

eCa 9y ago
In Perl5 there’s fc() which handles that. https://perldoc.perl.org/functions/fc.html https://perldoc.perl.org/functions/fc.html
- bzbarsky 9y ago
  "handles" in what sense? It needs to give different answers for the same letter depending on whether the text is English or Turkish, for example, so I don't see how it can "handle" it in general. Reading the documentation you link to, the answer in the fine print is that fc() gives you incorrect results for Turkish text.