5 ms·

I'm confused about why they even had the need for canonicalising the usernames for whatever purpose? Why couldn't they just have stored and used them just as th

I'm confused about why they even had the need for canonicalising the usernames for whatever purpose? Why couldn't they just have stored and used them just as they are?

dan_manges 13y ago
They gave one reason in the post. They wanted usernames to be case insensitive, so that if there's a user named BigBird, somebody else can't sign up as bigbird. Case insensitive usernames are also helpful to minimize support issues when somebody forgets the exact case they used when they created their account.
- drivebyacct2 13y ago
  Am I insensitive in wondering "Why not just limit it to lowercase, ASCII chars"?
  - EvanKelly 13y ago
    I don't think you're insensitive, but on a site which wants to foster users who care about their profile and persona on the site, freedom to display a username how they like could be a valuable feature.
  - mzl 13y ago
    Yes, because lowercase ASCII does not meaningfully capture how names are written in most of the world. If you want to cater to a global audience, it is simply not sufficient.
  - willvarfar 13y ago
    As they say in the article, they are trying to serve a global audience. As is easily googlable, Spotify was developed in the Baltic; the developer's own names likely contain non-ASCII characters.
    - drivebyacct2 13y ago
      Interesting, I felt foolish about an hour after having written this post, realizing that in most scenarios, it really shouldn't ever be any amount of effort minus half a second of planning and felt foolish for writing off unicode. (Normally I go the other route, unicode all the things). I suppose I feel that I was probably right to call myself arrogant then, to find out what you noted about Spotify's creators/creation. Interesting, thanks for the perspective check
      - oneeyedpigeon 13y ago
        I don't think you were all wrong - there's a difference between username and name. My name has a space in it, but they won't allow me to represent that in the username. And if my name is already used by someone else as their username, I can't use it as mine. This doesn't bother me, so long as my name is displayed properly in the UI (if at all).
yathern 13y ago
Because, then there can be multiple accounts that are variations of BigBird, and all annoying to differentiate for users. Such as Bigbird bigBird BigBIrd. Combind with this, the fact that you have unicode characters, and it makes it very easy to 'spoof' as another user by simply changing the letter 'o', to a unicode circle, or various other tricks.
jessaustin 13y ago
If they did as you suggest, the specific account-stealing flaw they had wouldn't happen, but since many unicode points have very similar glyphs, there would still be "copycat" accounts. That is, the strings "Oscar" and "Οscar" appear very similar (if one has the proper fonts installed), and one user could therefore pose as another. It's true that many sites don't care about this, but I don't fault Spotify for trying to prevent it.
- pilsetnieks 13y ago
  It looks to me as if the prevention of this caused more problems than it solved.
  - ars 13y ago
    Not even close. The letter À (A grave) can be written as the UTF-8 bytestream 0xC3 0x80 (i.e. a single "character"), or as À - i.e. a letter A, then a combining grave character i.e. 0x41 0xCC 0x80. The two are identical. Except they have different byte representations. If you don't normalize your unicode you will run into major problems.
    - jrochkind1 13y ago
      There are actually two kinds of normalization in play here. The one you are talking about is the one unicode actually calls normalization, and is dealt with in UTR#15. http://unicode.org/reports/tr15/ http://unicode.org/reports/tr15/ You are absolutely right that, in almost any situation taking unicode input where you're ever going to need to compare strings (and in most where you're ever going to need to display them), you are going to need to apply one of the UTR#15 normalization forms. UTR#15 normalizes different byte representations of what, in ALL circumstances are indeed identical characters/graphemes. A lot of people don't take account of this. Then there's the kind of canonicalization that OP talks about, which Unicode actually calls 'folding', and is about characters/graphemes which really ARE different characters but which, for _some_ but not all contexts may be treated as 'equivalent' (if not neccesarily identical). The simplest example is case insensitivity, but there are other trickier ones in the complete repertoire, like those discussed in the OP. This second kind of 'folding' canonicalization is a lot trickier, because it is contextual, not absolute. Which is maybe why Unicode started out trying to make an algorithm for it in UTR#30 but then abandoned it. Nonetheless, despite it's trickiness and contextuality, you often still really do need to do it, as in OP.

I'm confused about why they even had the need for canonicalising the usernames for whatever purpose? Why couldn't they just have stored and used them just as they are?

dan_manges 13y ago

They gave one reason in the post. They wanted usernames to be case insensitive, so that if there's a user named BigBird, somebody else can't sign up as bigbird. Case insensitive usernames are also helpful to minimize support issues when somebody forgets the exact case they used when they created their account.

drivebyacct2 13y ago
Am I insensitive in wondering "Why not just limit it to lowercase, ASCII chars"?
- EvanKelly 13y ago
  I don't think you're insensitive, but on a site which wants to foster users who care about their profile and persona on the site, freedom to display a username how they like could be a valuable feature.
- mzl 13y ago
  Yes, because lowercase ASCII does not meaningfully capture how names are written in most of the world. If you want to cater to a global audience, it is simply not sufficient.
- willvarfar 13y ago
  As they say in the article, they are trying to serve a global audience. As is easily googlable, Spotify was developed in the Baltic; the developer's own names likely contain non-ASCII characters.
  - drivebyacct2 13y ago
    Interesting, I felt foolish about an hour after having written this post, realizing that in most scenarios, it really shouldn't ever be any amount of effort minus half a second of planning and felt foolish for writing off unicode. (Normally I go the other route, unicode all the things). I suppose I feel that I was probably right to call myself arrogant then, to find out what you noted about Spotify's creators/creation. Interesting, thanks for the perspective check
    - oneeyedpigeon 13y ago
      I don't think you were all wrong - there's a difference between username and name. My name has a space in it, but they won't allow me to represent that in the username. And if my name is already used by someone else as their username, I can't use it as mine. This doesn't bother me, so long as my name is displayed properly in the UI (if at all).

yathern 13y ago

Because, then there can be multiple accounts that are variations of BigBird, and all annoying to differentiate for users. Such as Bigbird bigBird BigBIrd. Combind with this, the fact that you have unicode characters, and it makes it very easy to 'spoof' as another user by simply changing the letter 'o', to a unicode circle, or various other tricks.

jessaustin 13y ago

If they did as you suggest, the specific account-stealing flaw they had wouldn't happen, but since many unicode points have very similar glyphs, there would still be "copycat" accounts. That is, the strings "Oscar" and "Οscar" appear very similar (if one has the proper fonts installed), and one user could therefore pose as another. It's true that many sites don't care about this, but I don't fault Spotify for trying to prevent it.

pilsetnieks 13y ago
It looks to me as if the prevention of this caused more problems than it solved.
- ars 13y ago
  Not even close. The letter À (A grave) can be written as the UTF-8 bytestream 0xC3 0x80 (i.e. a single "character"), or as À - i.e. a letter A, then a combining grave character i.e. 0x41 0xCC 0x80. The two are identical. Except they have different byte representations. If you don't normalize your unicode you will run into major problems.
  - jrochkind1 13y ago
    There are actually two kinds of normalization in play here. The one you are talking about is the one unicode actually calls normalization, and is dealt with in UTR#15. http://unicode.org/reports/tr15/ http://unicode.org/reports/tr15/ You are absolutely right that, in almost any situation taking unicode input where you're ever going to need to compare strings (and in most where you're ever going to need to display them), you are going to need to apply one of the UTR#15 normalization forms. UTR#15 normalizes different byte representations of what, in ALL circumstances are indeed identical characters/graphemes. A lot of people don't take account of this. Then there's the kind of canonicalization that OP talks about, which Unicode actually calls 'folding', and is about characters/graphemes which really ARE different characters but which, for _some_ but not all contexts may be treated as 'equivalent' (if not neccesarily identical). The simplest example is case insensitivity, but there are other trickier ones in the complete repertoire, like those discussed in the OP. This second kind of 'folding' canonicalization is a lot trickier, because it is contextual, not absolute. Which is maybe why Unicode started out trying to make an algorithm for it in UTR#30 but then abandoned it. Nonetheless, despite it's trickiness and contextuality, you often still really do need to do it, as in OP.