5 ms·

This doesn't always work by the way. When you venture outside of ASCII, it's quite often uppercase(lowercase(x)) ≠ uppercase(x) and/or the other way around. Th

by foepys 5y ago

This doesn't always work by the way. When you venture outside of ASCII, it's quite often uppercase(lowercase(x)) ≠ uppercase(x) and/or the other way around. The German letter ß gets uppercased to SS instead of ẞ by most libraries in a neutral/generic culture. ẞ on the other hand gets lowercased to ß. This happens because there wasn't an official ẞ in German until recently but the uppercasing/lowercasing standard was already written for ß.

McMiniBurger 5y ago
but don't password fields only recognize ascii? it seems i just can't type korean to password fields
- jeroenhd 5y ago
  Most competent websites I know accept general UTF8 characters like emoji perfectly fine. There are a lot of crappier websites that don't even have proper unicode support for usernames or profile descriptions out there, though, so your mileage may vary. As far as I know, there's nothing preventing a password field from containing any valid unicode string. The problem may be IME support or servers stuck in ASCII, but the textbox itself will just work.
  - tsimionescu 5y ago
    Even surprisingly big names are surprisingly bad at this. Don't know recently, but Hotmail/Outlook used to have a rule of only using letters, numbers, and a handful of symbols, also limiting you to at most 16 characters or something. You couldn't even type a space!
    - jeroenhd 5y ago
      Honestly, that stuff only proves that big name websites aren't necessarily competent. PayPal used to let you register an account with a password longer than the maximum password length used in the authentication code, for example, essentially allowing you to set a password you could never use with your account again. Being worth billions doesn't mean you've got all the basics down, it just means you've tricked many people into giving you their business. Even good websites that will accept any valid password string will sometimes cut off the last part of a long password because their hashing algorithm throws that data away. Bcrypt, for example, supports a maximum input length between 50 and 72 bytes, depending on the library you use to hash your passwords. That's bytes, not characters! More primitive systems used to have problems with non-alfanumerical passwords and once those algorithms have been unleashed upon the unsuspecting public, you need to support them in your login flow for years to come.
    - jrootabega 5y ago
      They're not necessarily "bad" at it; there's a good chance they just want to make sure that the least competent of their users doesn't make a password that they have trouble with later. They don't care that security-conscious people get frustrated with it. So I guess that could also be "bad," but not incompetent "bad" or Michael Jackson "bad."
    - thefreeman 5y ago
      This is much more excusable for email providers to prevent phishing. There are a ton of unicode points that indistinguishable from ascii letters. There are other security issues that can arise as well. Here is an example from spotify https://engineering.atspotify.com/2013/06/18/creative-usernames/ https://engineering.atspotify.com/2013/06/18/creative-userna...
      - tsimionescu 5y ago
        I should have specified - this was (is?) for passwords, not usernames. I'm much more sympathetic to limited character sets in usernames, but I don't see much valid reason for doing so with passwords
    - spookthesunset 5y ago
      For what it’s worth the “big” company I work for stores usernames in MySQL. 15 years ago when the username column was created it was set for ASCII (or whatever legacy charset it was). Changing it to utf8 would be a royal pain in the ass, requiring all kinds of testing and crazy updates across the entire company. So while we’d love to make it utf8, it is just too much work to justify doing over other things.
      - tsimionescu 5y ago
        I should have noted - i was talking about their restrictions for passwords, not usernames. Since those are hashed before storage, i think there are far fewer excuses for such limitations.
  - deleted 5y ago
    [deleted]
skitter 5y ago
Another case where it doesn't work is with keyboard layouts that use shift lock instead of caps lock.
johannes1234321 5y ago
Thjs becomes even nkre fun with the Turkish i, which looks innocent, but doesn't necessarily become I. http://www.i18nguy.com/unicode/turkish-i18n.html http://www.i18nguy.com/unicode/turkish-i18n.html
naniwaduni 5y ago
This doesn't particularly matter since there's a perfectly good fallback for people who do happen to have "exotic" passwords: just type the password in correctly. As long as your mapping is consistent, it's totally fine if the bit sequence you're hashing is linguistically nonsense, because you're never going to display that to the user. What you don't normally want to do is normalize passwords before hashing and then only store the hash of the normalized string, because that's fragile to changes in your normalization algorithm, e.g. updating your Unicode data tables.

by foepys 5y ago

McMiniBurger 5y ago

but don't password fields only recognize ascii? it seems i just can't type korean to password fields

jeroenhd 5y ago
Most competent websites I know accept general UTF8 characters like emoji perfectly fine. There are a lot of crappier websites that don't even have proper unicode support for usernames or profile descriptions out there, though, so your mileage may vary. As far as I know, there's nothing preventing a password field from containing any valid unicode string. The problem may be IME support or servers stuck in ASCII, but the textbox itself will just work.
- tsimionescu 5y ago
  Even surprisingly big names are surprisingly bad at this. Don't know recently, but Hotmail/Outlook used to have a rule of only using letters, numbers, and a handful of symbols, also limiting you to at most 16 characters or something. You couldn't even type a space!
  - jeroenhd 5y ago
    Honestly, that stuff only proves that big name websites aren't necessarily competent. PayPal used to let you register an account with a password longer than the maximum password length used in the authentication code, for example, essentially allowing you to set a password you could never use with your account again. Being worth billions doesn't mean you've got all the basics down, it just means you've tricked many people into giving you their business. Even good websites that will accept any valid password string will sometimes cut off the last part of a long password because their hashing algorithm throws that data away. Bcrypt, for example, supports a maximum input length between 50 and 72 bytes, depending on the library you use to hash your passwords. That's bytes, not characters! More primitive systems used to have problems with non-alfanumerical passwords and once those algorithms have been unleashed upon the unsuspecting public, you need to support them in your login flow for years to come.
  - jrootabega 5y ago
    They're not necessarily "bad" at it; there's a good chance they just want to make sure that the least competent of their users doesn't make a password that they have trouble with later. They don't care that security-conscious people get frustrated with it. So I guess that could also be "bad," but not incompetent "bad" or Michael Jackson "bad."
  - thefreeman 5y ago
    This is much more excusable for email providers to prevent phishing. There are a ton of unicode points that indistinguishable from ascii letters. There are other security issues that can arise as well. Here is an example from spotify https://engineering.atspotify.com/2013/06/18/creative-usernames/ https://engineering.atspotify.com/2013/06/18/creative-userna...
    - tsimionescu 5y ago
      I should have specified - this was (is?) for passwords, not usernames. I'm much more sympathetic to limited character sets in usernames, but I don't see much valid reason for doing so with passwords
  - spookthesunset 5y ago
    For what it’s worth the “big” company I work for stores usernames in MySQL. 15 years ago when the username column was created it was set for ASCII (or whatever legacy charset it was). Changing it to utf8 would be a royal pain in the ass, requiring all kinds of testing and crazy updates across the entire company. So while we’d love to make it utf8, it is just too much work to justify doing over other things.
    - tsimionescu 5y ago
      I should have noted - i was talking about their restrictions for passwords, not usernames. Since those are hashed before storage, i think there are far fewer excuses for such limitations.
- deleted 5y ago
  [deleted]

skitter 5y ago

Another case where it doesn't work is with keyboard layouts that use shift lock instead of caps lock.

johannes1234321 5y ago

Thjs becomes even nkre fun with the Turkish i, which looks innocent, but doesn't necessarily become I. http://www.i18nguy.com/unicode/turkish-i18n.html http://www.i18nguy.com/unicode/turkish-i18n.html

naniwaduni 5y ago

This doesn't particularly matter since there's a perfectly good fallback for people who do happen to have "exotic" passwords: just type the password in correctly. As long as your mapping is consistent, it's totally fine if the bit sequence you're hashing is linguistically nonsense, because you're never going to display that to the user. What you don't normally want to do is normalize passwords before hashing and then only store the hash of the normalized string, because that's fragile to changes in your normalization algorithm, e.g. updating your Unicode data tables.