16 ms·

Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

370 points by _jg 7y ago

wolco 7y ago
So any email containing an i can be reset. Technically those using a custom domain name are immune but those using a general email service are at risk. Why would you write a general function that resets an account password but also accept an email address as a parameter? What use-case exists to change the email address sending the message?
ljm 7y ago
So if I understand this right, what GitHub did was something like: user = get_user_from_valid_email(params[:email]) send_reset_email(params[:email]) # instead of # send_reset_email(user.email) ? I've seen this pattern before and the reason is usually something about using the variable in memory as opposed to the function call. Total non-optimisation.
- _jg 7y ago
  Yes! Add in a toLowerCase() in there as well. An apparently small deviation with surprisingly large repercussions.
- microcolonel 7y ago
  We use three versions of the email address internally: the exact verified address used at signup or the last valid email change, a normalized version of that (for identity) without + mailboxes, lowercased, de-accented, stripped of dots and other inert punctuation, and normalized in a number of other ways... and then of course the email parameter (only used during registration). We accomplish this with a slightly more restrictive version of the standard ABNF provided in the RFC. I guess I should probably document why we go to this trouble, in case somebody gets the brilliant idea to "simplify" it.
  - _jg 7y ago
    Our team at Wisdom has done the exact same. It's been confusing at times, but much better once we got going.
  - _jal 7y ago
    > stripped of dots and other inert punctuation The period thing is a Gmail feature, not a standard. some.email@mydomain and someemail@mydomain most certainly do not deliver to the same mailbox.
    - microcolonel 7y ago
      > some.email@mydomain and someemail@mydomain most certainly do not deliver to the same mailbox. That's why we keep your verified mailbox address for sending mail; but there's no good reason to consider them different for the purpose of identity.
      - IAmEveryone 7y ago
        ...and only one in 50,000 e-mail addresses contain the string "rq5", therefore we strip that string from addresses... A false postive in these identity checks is likely to be less destructive than a false negative. But I still don't get the point of making up all sorts of rules not in the standard. I have seen both + as well as meaningful dots in e-mail adresses in the wild.
        _jal 7y ago
        + and . have is legal in email addresses since it was standardized. Google was the first installation I know of to silently swallow periods. Plus-addressing was well-known back in the day, but as far as I know GUI mailers more or less killed the practice by not offering support for it, and web sites written by people too smart to know how to validate email addresses ensured you can't even use them properly anymore. Ref: http://www.faqs.org/rfcs/rfc822.html http://www.faqs.org/rfcs/rfc822.html, pages 8/9.
      - aleph1 7y ago
        > That's why we keep your verified mailbox address for sending mail; but there's no good reason to consider them different for the purpose of identity. Both of Microsoft's own identity services (AAD and Live ID, and by extension O365 and Outlook.com) recognize (and allow creation of) microcolonel@example.com, micro.colonel@example.com, and mic.rocolonel@example.com as distinct identities/email addresses. What you're saying is that once the first of (microcolonel|micro.colonel|mic.rocolonel)@example.com registers at github, the other two will no longer be able to do so, but will instead receive a confusing 'you already have an account' error, (hopefully) without being able to receive password reset emails.
  - kevin_thibedeau 7y ago
    You can't assume that '+' has special meaning that can be stripped away.
    - microcolonel 7y ago
      Why not? On most major hosts it has a special meaning, and otherwise it is a relatively ridiculous thing to just add willy-nilly to your email address. We keep your verified mailbox address, the one you gave us, for sending mail. I doubt we'll ever turn away a customer by preventing registration of a new account sharing the prefix to a plus sign in their email address with an existing customer.
      - kevin_thibedeau 7y ago
        Senders don't get to dictate how a recipient encodes their addresses. RFC 822: The local-part of an addr-spec in a mailbox specification (i.e., the host's name for the mailbox) is understood to be whatever the receiving mail protocol server allows.
        microcolonel 7y ago
        I am well aware of that, but I'm comfortable requiring that new customers don't register an account with an email address foolishly designed to resemble another customer's email address in this particular way. We don't throw away their specified mailbox address, we just don't accept registrations which look suspiciously similar, or intended to cause confusion. I repeat, this has absolutely nothing to do with the mailbox address, where we send mail.
        kevin_thibedeau 7y ago
        You can't decide for yourself what the semantics of someone else's address do or do not mean.
        msandford 7y ago
  - dathinab 7y ago
    A lot of thinks you describe are Gmail properitary extensions. Especially the dot stripping. While far having two mail addresses only differing in dots is possible, especially with some older email addresses.
  - onion2k 7y ago
    It's very unlikely but still conceivable that robby@ and rob.by@ could be two different people at the same domain.
- oaiey 7y ago
  I always preferred using the record instead of the user input for follow up operations like that. So far it was only a gut feeling. Now I know why I do it ;)
- lazyjones 7y ago
  > Total non-optimisation. Lots of non-thought too. Sending e-mail directly from the place where web requests are processed isn't very smart. What if the SMTP subsystem is currently down or very slow? How many e-mails will you send if an attacker starts 1000 parallel web requests for a password reset? A saner way to do these things is to just set a flag ("password reset requested") in the user database there and do the actual work asynchronously in a regularly performed maintenance task. It'll also prevent all these attacks based on misuse of user input by default (unless, in this case, you pointlessly decide to update the user's e-mail in the user database to what the hacker specified).
  - londons_explore 7y ago
    Are designers like you the reason password reset mails frequently take 30 seconds plus to arrive? I expect things on the web to be damn near instant - I want to click that password reset button and hear a synchronous "ding" of an arriving mail. I don't want to wait for some cron job to run once per minute. If you want to send mail off the serving path, at least use a push based queue so there is no scheduling or polling involved.
    - c0l0 7y ago
      Yeah... and don't forget that the "R" in "SMTP" stands for "Real-time".
  - ljm 7y ago
    I can't speak for any other engineer anywhere else (where I'm sure they synchronously send out email because they haven't learned how to do it better), but there's not a chance github do this at their scale, and they're built on Rails. Rails gives you async mail by default and you just have to plug in a queue adapter for your worker processes to consume. e.g. GitHub gets this request, queues the job in redis/zero MQ/SQS or whatever they're using, and another process dedicated to sending those emails (or jobs with that priority) does the rest of the work. This is a massively common pattern in the Rails world and is as trivial to configure as your database connection.
- cryptonector 7y ago
  Or maybe someone wrote a validate_user_and_email() API and someone else wrote a send_password_reset_email() in a context where they lack access to the user DB, so they just validate the {username, email} and send to the attacker-provided email address. The first case (yours) is a plain bug, while the latter (mine) is an architecture bug. Architecture bugs often arise out of organizational bugs.
deathanatos 7y ago
This blog sets opacity: 0 (fully invisible) on the entire content, then fails to unset that CSS with JS, b/c the JS crashes if you block cookies. > because the system lowercased the provided email address and compared it to the email address stored in the user database. While sending the email to the attack-provided email, instead of the one in the database, is bad… lowercasing emails is also not valid. The lookup should never have matched in the first place. (It's slightly more complicated: to an extent, the case of the domain name doesn't matter, ignoring non-ASCII characters — I have no idea what they do. But the local part — the portion before the @ — is case sensitive. A server is free to ignore that, and map multiple local parts to the same mailbox internally¹, and many do, but the sender cannot make that assumption.) ¹or do other weird things, like ignore dots, or +extensions, etc.
- _jg 7y ago
  Unfortunately convention has normalised user expectations that email addresses are now case-insensitive. It's now a standard business requirement. Too bad few devs fully handle Unicode.
- glass_of_water 7y ago
  Nit: did you mean opacity: 0?
  - deathanatos 7y ago
    I did, thank you.
- jcranmer 7y ago
  Practically speaking, you have to treat email addresses as case-preserving: you match case-insensitive, but you always store the case the user entered in directly. I usually give my email address as starting with a capital letter for historical reasons, and I once had an issue that wouldn't let me log on with any case whatsoever, probably because it lowercased the email address and tried to match it against the uppercase in the database.
  - 7y ago
deleted 7y ago
[deleted]
notlukesky 7y ago
The likelihood of such a “hack” happening using the Turkish dotless “I” is ZERO as all Turkish email addresses and website domains are formatted WITHOUT using Turkish characters which include examples like: ç, ı, ü, ğ, ö, ş, İ, Ğ, Ü, Ö, Ş, Ç If you are at interested in Turkish characters: https://en.wikipedia.org/wiki/Wikipedia:Turkish_characters https://en.wikipedia.org/wiki/Wikipedia:Turkish_characters https://www.turkcebilgi.com/türkçe_karakter https://www.turkcebilgi.com/türkçe_karakter This should be called the Turkish character hack: A hack only possible in the theoretical realm. These characters are rendered in HTML and supported by OSes. They are just not used for emails AND website domains. So the password reset “hack” with an email containing Turkish characters in not a possibility from the get go. The whole attack vector hinges on emails that exist with Turkish characters in the first place.
- _jg 7y ago
  This was a very real and demonstrated vulnerability. Perhaps I've misunderstood your comment.
  - notlukesky 7y ago
    There are no emails with Turkish characters. The whole attack vector hinges on emails that exist with Turkish characters in the first place.
    - anderskaseorg 7y ago
      That’s incorrect. The attack vector hinges on the ability to create email addresses with Turkish characters. There is nothing stopping an attacker from creating addresses with Turkish characters to attack existing addresses without Turkish characters.
      - notlukesky 7y ago
        https://en.wikipedia.org/wiki/Email_address#Internationalization https://en.wikipedia.org/wiki/Email_address#Internationaliza... Turkish emails are not supported in the first place. Internationalization examples[edit] The example addresses below would not be handled by RFC 5322 based servers, but are permitted by RFC 6530. Servers compliant with this will be able to handle these: Latin alphabet with diacritics: Pelé@example.com Greek alphabet: δοκιμή@παράδειγμα.δοκιμή Traditional Chinese characters: 我買@屋企.香港 Japanese characters: 二ノ宮@黒川.日本 Cyrillic characters: медведь@с-балалайкой.рф Devanagari characters: संपर्क@डाटामेल.भारत
deleted 7y ago
[deleted]
fartcannon 7y ago
This is an organization that not only hosts a great deal of the worlds public/secret code, but is run by one of the largest data-gathering organizations on earth. I don't believe Microsoft can permit things like this under it's umbrella if it wants to continue to pretend that it's data-collection is benign.
michelpp 7y ago
The word "delıvered" is snuck into the article as a little Easter egg.
- Thorentis 7y ago
  So it is! It seems to even fool Chrome. If you search for "delivered" on the page the search box says "1/4" but entering will only take you to the 2 real ones, not the Turkish i ones which it has presumably counted.
  - Avamander 7y ago
    The alternative is worse though. Characters with umlauts matched with characters with no umlauts. E.g. searching for "rõõsa" will find both "rõõsa" and "roosa", incredibly annoying.
    - grenoire 7y ago
      Mind that an umlaut is the double dots (Ö, Ü, Ä, etc.) above letters; accents are any of the ones that are attached to the 'base set.'
veganjay 7y ago
There's a challenge in HackVent that involves unicode: http://whale.hacking-lab.com:8881/ http://whale.hacking-lab.com:8881/ I'm not sure if the solution is unicode collisions like in the article - still trying to solve it...
electrotype 7y ago
I have to admit this is an attack I wasn't aware of! I'll have to update some websites soon...
bowmessage 7y ago
Author: any word on how much GitHub paid out for discovering this vulnerability? Also, how simple was it to create a unicode-based email address on one of the large providers?
jrochkind1 7y ago
The examples in the initial "quick example" are backwards, no? It's `'ß'.toUpperCase()` that is `"SS"`, not `'ß'.toLowerCase() === 'ss'`. As the later chart makes clear. Same with turkish ı.
- dyu 7y ago
  I believe you are correct. However, this particular example may not be the best one, see: https://en.wikipedia.org/wiki/Capital_%E1%BA%9E#Current_situation https://en.wikipedia.org/wiki/Capital_%E1%BA%9E#Current_situ...
- gitgud 7y ago
  Well I just tried in the browser console and got: 'ß'.toUpperCase() // = "SS" 'ß'.toLowerCase() // = "ß"
  - dathinab 7y ago
    Note that Unicode did add a "uppercase-ish" ß, as it does appear in German, but only in context of an all caps word e.g on a sign board, so captilazation of a whole word to SS and that new all caps ß are both correct (not sure if UNICODE changed the capitalization rules or just added that strange all caps ß)
    - thristian 7y ago
      For backwards compatibility reasons, the capitalization rules can't be changed for existing characters. So normalizing by naive case-folding now requires at least three steps: "ẞ".to_lower() → "ß" "ß".to_upper() → "SS" "SS".to_lower() → "ss" (there's a standard for how to compare strings case-insensitively that doesn't involve repeated case-folding, but it's much more complex)
      - jrochkind1 7y ago
        Wow, I did not know about this weird edge case, interesting, thanks!
BoppreH 7y ago
I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical file names may map to different files (because confusables[1]), or two different names map to the same file (because normalization[2]), or the ".jpg" at the end may not actually be the extension (because right-to-left override[3]), not to mention names with newlines or backspaces in them, and inconsistencies between operating systems. I would go as far as blaming our overreliance on strings for all the injection attacks we see (XSS, SQL, command, etc). [1] https://unicode.org/cldr/utility/confusables.jsp https://unicode.org/cldr/utility/confusables.jsp [2] https://developer.apple.com/library/archive/qa/qa1173/_index.html https://developer.apple.com/library/archive/qa/qa1173/_index... [3] https://krebsonsecurity.com/2011/09/right-to-left-override-aids-email-attacks/ https://krebsonsecurity.com/2011/09/right-to-left-override-a...
- kortex 7y ago
  How do you do the rendering? Who (process-wise) is responsible for converting bytes to pixels? How do users on social media put in their name? How is it stored? How do users get urls with specific usernames? Now take all that and multiply by the complexity of world languages, many which don't even map to one glyph == one morpheme. The ol' apple message crash bug was due to the property of some Arabic not being monotonic in rendering space vs string length. I think we could have skipped utf8 and just gone to 4byte runes. But even then, that would not have avoided the above bug. Utf16 is a hot mess though, worst of all worlds.
  - BoppreH 7y ago
    The problem is now deeply entrenched, so I don't have perfect answers. But here are my guesses: > Who (process-wise) is responsible for converting bytes to pixels? The operating system, with minor exceptions (word processors, for example). Rendering logic is too complicated to be embedded into every application. And you get better accessibility and consistency. > How do users on social media put in their name? How is it stored? Keyboards, or their preferred input methods, and stored in UTF-8. I'm not saying to get rid of all strings, just don't use it for infrastructure. > How do users get urls with specific usernames? You don't, because that's how you get little Bobby FRACTION-SLASH. Also, if you have a valuable namespace like URLs, people will hack each other to get valuable names, but that's only tangentially related. > Now take all that and multiply by the complexity of world languages, many which don't even map to one glyph == one morpheme. The ol' apple message crash bug was due to the property of some Arabic not being monotonic in rendering space vs string length. That's exactly my point! You get this multiplied complexity when people try to peek into string contents instead of treating them like black boxes. Stop with the dangerous string operations and you now support usernames with zalgo-ed hieroglyphs if that's what users want. > I think we could have skipped utf8 and just gone to 4byte runes. But even then, that would not have avoided the above bug. > Utf16 is a hot mess though, worst of all worlds. Agreed with UTF-16. I like UTF-8, and I honestly think it solved our encoding problems for non-legacy applications. Everyone should be using it, as long as the contents are for human consumption only.
kerng 7y ago
What is so fascinating about security is that the same problems keep popping up again once in a while - I remember the Turkish i issues being a big problem in early mid 2000s during all the security pushes that went through software engineering world. Then we kinda forgot about them, and now it keeps coming up again. The best a researcher can do is go back 20+ years and look for security issue that occurred back then. Most likely you'll find very similar things now again.
gumby 7y ago
Unicode actually has an uppercase ß though I don't understand why.
- _emacsomancer_ 7y ago
  For typesetting in allcaps.
- thewarpaint 7y ago
  Because it's part of the German language: https://en.m.wikipedia.org/wiki/Capital_%E1%BA%9E https://en.m.wikipedia.org/wiki/Capital_%E1%BA%9E
  - gumby 7y ago
    Barely. If you look at it it is clearly just what its name says: a lowercase medial s combined with a lowercase z. The uppercase version exists, as a parallel commentator noted, basically as a typographic utility. Fraktur had/has other such lower case ligatures (tz, ch, sch, ss (not ß) et al) but for some reason only ß survived into Latin script as a full fledged letter. I have a lot of old (mostly 20th century) books in Fraktur and they all use these ligatures more consistently than the Latin ligatures are used.
    - 80386 7y ago
      Fraktur ß is a ligature of s and z; the current form came into use in Latin-script German because the Latin script already had ß, for the ligature of s and s. Some sort of orthographic device is needed - s between two vowels means the first vowel is long and the consonant is voiced, and ss between two vowels means the first vowel is short and the consonant is voiceless. So it's useful to have a different case for when the first vowel is long and the consonant is voiceless: Busen /bu:zən/ Busse /busə/ Buße /buːsə/ Vowel length isn't predictable from spelling in cases of consonant clusters and other digraphs: Hand /hant/ vs. Mond /moːnt/, Bruch /brux/ vs. Buch /buːx/, etc. So ss could've been used for both - or sz, although there are some words spelled with sz as a sequence of s and z, like Szene /stseːnə/.
mrighele 7y ago
> // Note the Turkish dotless i 'John@Gıthub.com'.toLowerCase() === 'John@Github.com'.toLowerCase() I'm not sure this example is correct. The dotless ı is already lower cased, so the comparison above should yield false. Maybe the author was thinking about upper case dotted "İ", which becomes regular dotted "i" when lower cased. So what could happen is that an user enter "JOHN@GİTHUB.COM" as email, and then the email is sent to "john@github.com" .
- _jg 7y ago
  Should be: 'John@Gıthub.com'.toUpperCase() === 'John@Github.com'.toUpperCase()
  - stedaniels 7y ago
    The example doesn't seem to work at all for GitHub's explanation.. they say that their outgoing email server didn't support unicode in the domain part anyway. What am I missing? Was your actual attack on the local part?
blipmusic 7y ago
Julia returns: julia> c='ı' 'ı': Unicode U+0131 (category Ll: Letter, lowercase) julia> uppercase(c) 'I': ASCII/Unicode U+0049 (category Lu: Letter, uppercase) julia> lowercase(uppercase(c)) 'i': ASCII/Unicode U+0069 (category Ll: Letter, lowercase) Is this something that needs changing in the Unicode spec itself or how strings are handled in general by various tools/programming languages? I love [plain] text, but it's so, so fragile. :/
- ishi 7y ago
  I believe this specific problem exists in Unicode, and not just in the Julia language.
dathinab 7y ago
As a side note puny code conversion is only defined for the domain name _not_ the local part. Using puny code on the local part will semantically create a different email and at last theoretically a mail provider might support both the puny code and normal version as two different mail addreses and as such using punicode there would potentially open up a different vulnarability. Now that I think about it as far as I remember the local part of mail is actually not defined as cases insensitive , through all? mail programs treat it as such. The important part her is to always use data from your database for any security relevant parts.
leowoo91 7y ago
Just checked gmaıl IDN as xn--gmal-nza.com which is registered by: 2019-02-27
hashStuff 7y ago
Wouldn't having the email hashed prevent this? Not sure why email addresses are stored plain text still. Especially if they are stored in databases.
- sophiebits 7y ago
  Emails are used for, among other things, sending email to a user. Which requires the application know the email address.
hashStuff 7y ago
Would this not have been solved if the email addresses were stored as hashes? Besides this it would be an extra layer of security in the even of a breach. Why addresses aren't stored as hashes seems silly. Especially when stored in true databases that can be queried quickly.
Keverw 7y ago
Reminds me of a issue Spotify had too with unicodes and usernames. https://labs.spotify.com/2013/06/18/creative-usernames/ https://labs.spotify.com/2013/06/18/creative-usernames/ I had someone tell me that programming isn't real work before, and this is yet another example of all the small little details going into building things that most people don't really think about day to day. I haven't had to work with login code in a while, but might at some point. I know some systems only allow alphanumeric usernames, but sounds like can't force people to have alphanumeric emails... Well I guess you could but might upset someone. I know there's normalizing functions though like NFD, NFC, NFKD or NFKC that might work for usernames but not sure what's really recommended. Then also brute forcing attempts to try to mitigate attacks and other considerations to make also when building out an account system. Then if your company is large enough to provide phone support, not sure how you'd tell the support person which specific emoji you used in your username.
- keyP 7y ago
  > I had someone tell me that programming isn't real work before Haha, I don't even understand what metrics the person was using to consider something "work". A pilot sits throughout the flight but I think it's fair to say their working...
  - Keverw 7y ago
    Yeah. I guess this person thinks sitting at a computer all day isn't real work. Real work would be working in a factory all your life breaking your back. Then I also think some people think making websites and coding is the same as using Word or Powerpoint. I guess they just don't really understand tech, probably a lot of people in the rust belt. Probably why they're driving young people away.
inimino 7y ago
Domains are not case sensitive, but email local parts are! There is no reason whatsoever to do case normalization on local parts of emails on any domain you do not own, as this is strictly incorrect and could lead to a totally different address that also exists (as happened here). Of course, email providers are free to do whatever case folding or normalization they want, in which case the security burden of avoiding collisions is on the provider. If someone's email provider maps different case variants to the same mailbox, there's still no need whatsoever to do anything to the address, as the user will get it delivered to them regardless. If the provider doesn't do case folding, they will have to enter their local part case sensitively, but that's exactly the same as for any other use of their email address. I can only imagine how this vulnerability came to be. Unicode is not to blame here. If security-critical password reset code was not audited carefully enough to catch a mistake like this, one wonders what other errors might remain.
evilotto 7y ago
That's the same glyph that got a guy killed. https://gizmodo.com/a-cellphones-missing-dot-kills-two-people-puts-three-m-382026 https://gizmodo.com/a-cellphones-missing-dot-kills-two-peopl...
- mrmcd 7y ago
  My girlfriend is from Turkey, and I shared this story with her a while back (I discovered it while also researching some Unicode collision issues.) She said that while it's an entertaining story, there's almost certainly a bit of sensationalism and exaggeration from the Turkish press combined with credulity by English language journalists when re-reporting it. Basically, the supposed texts wouldn't have made sense in terms of grammar and syntax with a straightforward dotless/dotted I swap, and would've been obvious to someone fluent in Turkish what happened. This is would've been especially true if you had had this cell phone for any amount of time and communicating in Turkish and it had been routinely swapping Is. More likely is a bunch of young and/or not too bright people were looking for a reason to get into a violent confrontation. Then the muckraking Turkish press had a sensationalist murder-suicide lovers quarrel story, and as a bonus a nationalistic "see how cell phone companies don't respect our culture" angle as the cherry on top. However, you should still ALWAYS be careful when converting between character sets and be locality aware when manipulating strings. Practice string safety. ;)
qwerty456127 7y ago
> 'ß'.toLowerCase() // 'ss' Why? First of all ß is already lowercase, why should toLowerCase() change it? It also is a normal letter having both uppercase (ẞ) and lowercase (ß) forms so converting between the cases can be made be trivial and quirk-free. Arguably the most common word you will encounter ß in is Straße (street) where it already is lowercase - will "Straße".toLowerCase() turn it into "strasse"? WTF? "Straße".lower() returns "straße" in Python which seems reasonable (nevertheless "Straße".upper() actually returns "STRASSE" ignoring the existence of the uppercase ẞ (U+1E9E)). Why should it behave different in JavaScript? (Because JavaScript is different, I know, just a rhetorical question)
- enlyth 7y ago
  I must be missing something because I tried three browsers and all their JavaScript engines evaluate 'ß'.toLowerCase() as 'ß', not 'ss'.
- tomcooks 7y ago
  "We are living in America, Amerika ist wunderbar"
- thaumasiotes 7y ago
  > (nevertheless "Straße".upper() actually returns "STRASSE" ignoring the existence of the uppercase ẞ (U+1E9E)) Python probably predates the addition of a notional capital ß glyph in 2017. SS is the capitalization of ß that you'd expect if you were thinking of your data as a string rather than a collection of font elements.
ptah 7y ago
> Have we convinced you that Unicode is Awesome? this is sarcasm right?
MattConfluence 7y ago
The linked post is titled "Hacking GitHub with Unicode's dotless 'i'.", but this submission is in title case "Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'". I think this is a bad title change, because an uppercase I is supposed to be dotless, whereas the lowercase i used by the author's title is not.
reportgunner 7y ago
I haven't finished reading the article, but isn't the problem here that they are sending the e-mail to the address provided by the user rather than sending it to the e-mail stored in the database ? I fail to see any added value in sending a reset link to an e-mail entered by the user (while that e-mail is already in the database). Is it because the e-mails are stored hashed ?
angst 7y ago
>'ß'.toLowerCase() // 'ss' "ß" >'ß'.toLowerCase() === 'SS'.toLowerCase() // true false >// Note the Turkish dotless i >'John@Gıthub.com'.toUpperCase() === 'John@Github.com'.toUpperCase() true Chrome 79.0.3945.79 (Windows, 64bit) seems to differ from the proposed results for the first two statements. If the proposed results are what should indeed happen by unicode standard then I wonder chrome is not fully implementing them?
lamby 7y ago
Django has announced their security releases to match. [0] [0] https://www.djangoproject.com/weblog/2019/dec/18/security-releases/ https://www.djangoproject.com/weblog/2019/dec/18/security-re...

370 points by _jg 7y ago

So any email containing an i can be reset. Technically those using a custom domain name are immune but those using a general email service are at risk. Why would you write a general function that resets an account password but also accept an email address as a parameter? What use-case exists to change the email address sending the message?

ljm 7y ago

So if I understand this right, what GitHub did was something like: user = get_user_from_valid_email(params[:email]) send_reset_email(params[:email]) # instead of # send_reset_email(user.email) ? I've seen this pattern before and the reason is usually something about using the variable in memory as opposed to the function call. Total non-optimisation.

_jg 7y ago
Yes! Add in a toLowerCase() in there as well. An apparently small deviation with surprisingly large repercussions.
microcolonel 7y ago
We use three versions of the email address internally: the exact verified address used at signup or the last valid email change, a normalized version of that (for identity) without + mailboxes, lowercased, de-accented, stripped of dots and other inert punctuation, and normalized in a number of other ways... and then of course the email parameter (only used during registration). We accomplish this with a slightly more restrictive version of the standard ABNF provided in the RFC. I guess I should probably document why we go to this trouble, in case somebody gets the brilliant idea to "simplify" it.
- _jg 7y ago
  Our team at Wisdom has done the exact same. It's been confusing at times, but much better once we got going.
- _jal 7y ago
  > stripped of dots and other inert punctuation The period thing is a Gmail feature, not a standard. some.email@mydomain and someemail@mydomain most certainly do not deliver to the same mailbox.
  - microcolonel 7y ago
    > some.email@mydomain and someemail@mydomain most certainly do not deliver to the same mailbox. That's why we keep your verified mailbox address for sending mail; but there's no good reason to consider them different for the purpose of identity.
    - IAmEveryone 7y ago
      ...and only one in 50,000 e-mail addresses contain the string "rq5", therefore we strip that string from addresses... A false postive in these identity checks is likely to be less destructive than a false negative. But I still don't get the point of making up all sorts of rules not in the standard. I have seen both + as well as meaningful dots in e-mail adresses in the wild.
      - _jal 7y ago
        + and . have is legal in email addresses since it was standardized. Google was the first installation I know of to silently swallow periods. Plus-addressing was well-known back in the day, but as far as I know GUI mailers more or less killed the practice by not offering support for it, and web sites written by people too smart to know how to validate email addresses ensured you can't even use them properly anymore. Ref: http://www.faqs.org/rfcs/rfc822.html http://www.faqs.org/rfcs/rfc822.html, pages 8/9.
    - aleph1 7y ago
      > That's why we keep your verified mailbox address for sending mail; but there's no good reason to consider them different for the purpose of identity. Both of Microsoft's own identity services (AAD and Live ID, and by extension O365 and Outlook.com) recognize (and allow creation of) microcolonel@example.com, micro.colonel@example.com, and mic.rocolonel@example.com as distinct identities/email addresses. What you're saying is that once the first of (microcolonel|micro.colonel|mic.rocolonel)@example.com registers at github, the other two will no longer be able to do so, but will instead receive a confusing 'you already have an account' error, (hopefully) without being able to receive password reset emails.
- kevin_thibedeau 7y ago
  You can't assume that '+' has special meaning that can be stripped away.
  - microcolonel 7y ago
    Why not? On most major hosts it has a special meaning, and otherwise it is a relatively ridiculous thing to just add willy-nilly to your email address. We keep your verified mailbox address, the one you gave us, for sending mail. I doubt we'll ever turn away a customer by preventing registration of a new account sharing the prefix to a plus sign in their email address with an existing customer.
    - kevin_thibedeau 7y ago
      Senders don't get to dictate how a recipient encodes their addresses. RFC 822: The local-part of an addr-spec in a mailbox specification (i.e., the host's name for the mailbox) is understood to be whatever the receiving mail protocol server allows.
      - microcolonel 7y ago
        I am well aware of that, but I'm comfortable requiring that new customers don't register an account with an email address foolishly designed to resemble another customer's email address in this particular way. We don't throw away their specified mailbox address, we just don't accept registrations which look suspiciously similar, or intended to cause confusion. I repeat, this has absolutely nothing to do with the mailbox address, where we send mail.
        kevin_thibedeau 7y ago
        You can't decide for yourself what the semantics of someone else's address do or do not mean.
        msandford 7y ago
- dathinab 7y ago
  A lot of thinks you describe are Gmail properitary extensions. Especially the dot stripping. While far having two mail addresses only differing in dots is possible, especially with some older email addresses.
- onion2k 7y ago
  It's very unlikely but still conceivable that robby@ and rob.by@ could be two different people at the same domain.
oaiey 7y ago
I always preferred using the record instead of the user input for follow up operations like that. So far it was only a gut feeling. Now I know why I do it ;)
lazyjones 7y ago
> Total non-optimisation. Lots of non-thought too. Sending e-mail directly from the place where web requests are processed isn't very smart. What if the SMTP subsystem is currently down or very slow? How many e-mails will you send if an attacker starts 1000 parallel web requests for a password reset? A saner way to do these things is to just set a flag ("password reset requested") in the user database there and do the actual work asynchronously in a regularly performed maintenance task. It'll also prevent all these attacks based on misuse of user input by default (unless, in this case, you pointlessly decide to update the user's e-mail in the user database to what the hacker specified).
- londons_explore 7y ago
  Are designers like you the reason password reset mails frequently take 30 seconds plus to arrive? I expect things on the web to be damn near instant - I want to click that password reset button and hear a synchronous "ding" of an arriving mail. I don't want to wait for some cron job to run once per minute. If you want to send mail off the serving path, at least use a push based queue so there is no scheduling or polling involved.
  - c0l0 7y ago
    Yeah... and don't forget that the "R" in "SMTP" stands for "Real-time".
- ljm 7y ago
  I can't speak for any other engineer anywhere else (where I'm sure they synchronously send out email because they haven't learned how to do it better), but there's not a chance github do this at their scale, and they're built on Rails. Rails gives you async mail by default and you just have to plug in a queue adapter for your worker processes to consume. e.g. GitHub gets this request, queues the job in redis/zero MQ/SQS or whatever they're using, and another process dedicated to sending those emails (or jobs with that priority) does the rest of the work. This is a massively common pattern in the Rails world and is as trivial to configure as your database connection.
cryptonector 7y ago
Or maybe someone wrote a validate_user_and_email() API and someone else wrote a send_password_reset_email() in a context where they lack access to the user DB, so they just validate the {username, email} and send to the attacker-provided email address. The first case (yours) is a plain bug, while the latter (mine) is an architecture bug. Architecture bugs often arise out of organizational bugs.

deathanatos 7y ago

This blog sets opacity: 0 (fully invisible) on the entire content, then fails to unset that CSS with JS, b/c the JS crashes if you block cookies. > because the system lowercased the provided email address and compared it to the email address stored in the user database. While sending the email to the attack-provided email, instead of the one in the database, is bad… lowercasing emails is also not valid. The lookup should never have matched in the first place. (It's slightly more complicated: to an extent, the case of the domain name doesn't matter, ignoring non-ASCII characters — I have no idea what they do. But the local part — the portion before the @ — is case sensitive. A server is free to ignore that, and map multiple local parts to the same mailbox internally¹, and many do, but the sender cannot make that assumption.) ¹or do other weird things, like ignore dots, or +extensions, etc.

_jg 7y ago
Unfortunately convention has normalised user expectations that email addresses are now case-insensitive. It's now a standard business requirement. Too bad few devs fully handle Unicode.
glass_of_water 7y ago
Nit: did you mean opacity: 0?
- deathanatos 7y ago
  I did, thank you.
jcranmer 7y ago
Practically speaking, you have to treat email addresses as case-preserving: you match case-insensitive, but you always store the case the user entered in directly. I usually give my email address as starting with a capital letter for historical reasons, and I once had an issue that wouldn't let me log on with any case whatsoever, probably because it lowercased the email address and tried to match it against the uppercase in the database.
- 7y ago

deleted 7y ago

[deleted]

notlukesky 7y ago

The likelihood of such a “hack” happening using the Turkish dotless “I” is ZERO as all Turkish email addresses and website domains are formatted WITHOUT using Turkish characters which include examples like: ç, ı, ü, ğ, ö, ş, İ, Ğ, Ü, Ö, Ş, Ç If you are at interested in Turkish characters: https://en.wikipedia.org/wiki/Wikipedia:Turkish_characters https://en.wikipedia.org/wiki/Wikipedia:Turkish_characters https://www.turkcebilgi.com/türkçe_karakter https://www.turkcebilgi.com/türkçe_karakter This should be called the Turkish character hack: A hack only possible in the theoretical realm. These characters are rendered in HTML and supported by OSes. They are just not used for emails AND website domains. So the password reset “hack” with an email containing Turkish characters in not a possibility from the get go. The whole attack vector hinges on emails that exist with Turkish characters in the first place.

_jg 7y ago
This was a very real and demonstrated vulnerability. Perhaps I've misunderstood your comment.
- notlukesky 7y ago
  There are no emails with Turkish characters. The whole attack vector hinges on emails that exist with Turkish characters in the first place.
  - anderskaseorg 7y ago
    That’s incorrect. The attack vector hinges on the ability to create email addresses with Turkish characters. There is nothing stopping an attacker from creating addresses with Turkish characters to attack existing addresses without Turkish characters.
    - notlukesky 7y ago
      https://en.wikipedia.org/wiki/Email_address#Internationalization https://en.wikipedia.org/wiki/Email_address#Internationaliza... Turkish emails are not supported in the first place. Internationalization examples[edit] The example addresses below would not be handled by RFC 5322 based servers, but are permitted by RFC 6530. Servers compliant with this will be able to handle these: Latin alphabet with diacritics: Pelé@example.com Greek alphabet: δοκιμή@παράδειγμα.δοκιμή Traditional Chinese characters: 我買@屋企.香港 Japanese characters: 二ノ宮@黒川.日本 Cyrillic characters: медведь@с-балалайкой.рф Devanagari characters: संपर्क@डाटामेल.भारत

deleted 7y ago

[deleted]

fartcannon 7y ago

This is an organization that not only hosts a great deal of the worlds public/secret code, but is run by one of the largest data-gathering organizations on earth. I don't believe Microsoft can permit things like this under it's umbrella if it wants to continue to pretend that it's data-collection is benign.

michelpp 7y ago

The word "delıvered" is snuck into the article as a little Easter egg.

Thorentis 7y ago
So it is! It seems to even fool Chrome. If you search for "delivered" on the page the search box says "1/4" but entering will only take you to the 2 real ones, not the Turkish i ones which it has presumably counted.
- Avamander 7y ago
  The alternative is worse though. Characters with umlauts matched with characters with no umlauts. E.g. searching for "rõõsa" will find both "rõõsa" and "roosa", incredibly annoying.
  - grenoire 7y ago
    Mind that an umlaut is the double dots (Ö, Ü, Ä, etc.) above letters; accents are any of the ones that are attached to the 'base set.'

veganjay 7y ago

There's a challenge in HackVent that involves unicode: http://whale.hacking-lab.com:8881/ http://whale.hacking-lab.com:8881/ I'm not sure if the solution is unicode collisions like in the article - still trying to solve it...

electrotype 7y ago

I have to admit this is an attack I wasn't aware of! I'll have to update some websites soon...

bowmessage 7y ago

Author: any word on how much GitHub paid out for discovering this vulnerability? Also, how simple was it to create a unicode-based email address on one of the large providers?

jrochkind1 7y ago

The examples in the initial "quick example" are backwards, no? It's `'ß'.toUpperCase()` that is `"SS"`, not `'ß'.toLowerCase() === 'ss'`. As the later chart makes clear. Same with turkish ı.

dyu 7y ago
I believe you are correct. However, this particular example may not be the best one, see: https://en.wikipedia.org/wiki/Capital_%E1%BA%9E#Current_situation https://en.wikipedia.org/wiki/Capital_%E1%BA%9E#Current_situ...
gitgud 7y ago
Well I just tried in the browser console and got: 'ß'.toUpperCase() // = "SS" 'ß'.toLowerCase() // = "ß"
- dathinab 7y ago
  Note that Unicode did add a "uppercase-ish" ß, as it does appear in German, but only in context of an all caps word e.g on a sign board, so captilazation of a whole word to SS and that new all caps ß are both correct (not sure if UNICODE changed the capitalization rules or just added that strange all caps ß)
  - thristian 7y ago
    For backwards compatibility reasons, the capitalization rules can't be changed for existing characters. So normalizing by naive case-folding now requires at least three steps: "ẞ".to_lower() → "ß" "ß".to_upper() → "SS" "SS".to_lower() → "ss" (there's a standard for how to compare strings case-insensitively that doesn't involve repeated case-folding, but it's much more complex)
    - jrochkind1 7y ago
      Wow, I did not know about this weird edge case, interesting, thanks!

BoppreH 7y ago

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical file names may map to different files (because confusables[1]), or two different names map to the same file (because normalization[2]), or the ".jpg" at the end may not actually be the extension (because right-to-left override[3]), not to mention names with newlines or backspaces in them, and inconsistencies between operating systems. I would go as far as blaming our overreliance on strings for all the injection attacks we see (XSS, SQL, command, etc). [1] https://unicode.org/cldr/utility/confusables.jsp https://unicode.org/cldr/utility/confusables.jsp [2] https://developer.apple.com/library/archive/qa/qa1173/_index.html https://developer.apple.com/library/archive/qa/qa1173/_index... [3] https://krebsonsecurity.com/2011/09/right-to-left-override-aids-email-attacks/ https://krebsonsecurity.com/2011/09/right-to-left-override-a...

kortex 7y ago
How do you do the rendering? Who (process-wise) is responsible for converting bytes to pixels? How do users on social media put in their name? How is it stored? How do users get urls with specific usernames? Now take all that and multiply by the complexity of world languages, many which don't even map to one glyph == one morpheme. The ol' apple message crash bug was due to the property of some Arabic not being monotonic in rendering space vs string length. I think we could have skipped utf8 and just gone to 4byte runes. But even then, that would not have avoided the above bug. Utf16 is a hot mess though, worst of all worlds.
- BoppreH 7y ago
  The problem is now deeply entrenched, so I don't have perfect answers. But here are my guesses: > Who (process-wise) is responsible for converting bytes to pixels? The operating system, with minor exceptions (word processors, for example). Rendering logic is too complicated to be embedded into every application. And you get better accessibility and consistency. > How do users on social media put in their name? How is it stored? Keyboards, or their preferred input methods, and stored in UTF-8. I'm not saying to get rid of all strings, just don't use it for infrastructure. > How do users get urls with specific usernames? You don't, because that's how you get little Bobby FRACTION-SLASH. Also, if you have a valuable namespace like URLs, people will hack each other to get valuable names, but that's only tangentially related. > Now take all that and multiply by the complexity of world languages, many which don't even map to one glyph == one morpheme. The ol' apple message crash bug was due to the property of some Arabic not being monotonic in rendering space vs string length. That's exactly my point! You get this multiplied complexity when people try to peek into string contents instead of treating them like black boxes. Stop with the dangerous string operations and you now support usernames with zalgo-ed hieroglyphs if that's what users want. > I think we could have skipped utf8 and just gone to 4byte runes. But even then, that would not have avoided the above bug. > Utf16 is a hot mess though, worst of all worlds. Agreed with UTF-16. I like UTF-8, and I honestly think it solved our encoding problems for non-legacy applications. Everyone should be using it, as long as the contents are for human consumption only.

kerng 7y ago

What is so fascinating about security is that the same problems keep popping up again once in a while - I remember the Turkish i issues being a big problem in early mid 2000s during all the security pushes that went through software engineering world. Then we kinda forgot about them, and now it keeps coming up again. The best a researcher can do is go back 20+ years and look for security issue that occurred back then. Most likely you'll find very similar things now again.

gumby 7y ago

Unicode actually has an uppercase ß though I don't understand why.

_emacsomancer_ 7y ago
For typesetting in allcaps.
thewarpaint 7y ago
Because it's part of the German language: https://en.m.wikipedia.org/wiki/Capital_%E1%BA%9E https://en.m.wikipedia.org/wiki/Capital_%E1%BA%9E
- gumby 7y ago
  Barely. If you look at it it is clearly just what its name says: a lowercase medial s combined with a lowercase z. The uppercase version exists, as a parallel commentator noted, basically as a typographic utility. Fraktur had/has other such lower case ligatures (tz, ch, sch, ss (not ß) et al) but for some reason only ß survived into Latin script as a full fledged letter. I have a lot of old (mostly 20th century) books in Fraktur and they all use these ligatures more consistently than the Latin ligatures are used.
  - 80386 7y ago
    Fraktur ß is a ligature of s and z; the current form came into use in Latin-script German because the Latin script already had ß, for the ligature of s and s. Some sort of orthographic device is needed - s between two vowels means the first vowel is long and the consonant is voiced, and ss between two vowels means the first vowel is short and the consonant is voiceless. So it's useful to have a different case for when the first vowel is long and the consonant is voiceless: Busen /bu:zən/ Busse /busə/ Buße /buːsə/ Vowel length isn't predictable from spelling in cases of consonant clusters and other digraphs: Hand /hant/ vs. Mond /moːnt/, Bruch /brux/ vs. Buch /buːx/, etc. So ss could've been used for both - or sz, although there are some words spelled with sz as a sequence of s and z, like Szene /stseːnə/.

mrighele 7y ago

> // Note the Turkish dotless i 'John@Gıthub.com'.toLowerCase() === 'John@Github.com'.toLowerCase() I'm not sure this example is correct. The dotless ı is already lower cased, so the comparison above should yield false. Maybe the author was thinking about upper case dotted "İ", which becomes regular dotted "i" when lower cased. So what could happen is that an user enter "JOHN@GİTHUB.COM" as email, and then the email is sent to "john@github.com" .

_jg 7y ago
Should be: 'John@Gıthub.com'.toUpperCase() === 'John@Github.com'.toUpperCase()
- stedaniels 7y ago
  The example doesn't seem to work at all for GitHub's explanation.. they say that their outgoing email server didn't support unicode in the domain part anyway. What am I missing? Was your actual attack on the local part?

blipmusic 7y ago

Julia returns: julia> c='ı' 'ı': Unicode U+0131 (category Ll: Letter, lowercase) julia> uppercase(c) 'I': ASCII/Unicode U+0049 (category Lu: Letter, uppercase) julia> lowercase(uppercase(c)) 'i': ASCII/Unicode U+0069 (category Ll: Letter, lowercase) Is this something that needs changing in the Unicode spec itself or how strings are handled in general by various tools/programming languages? I love [plain] text, but it's so, so fragile. :/

ishi 7y ago
I believe this specific problem exists in Unicode, and not just in the Julia language.

dathinab 7y ago

As a side note puny code conversion is only defined for the domain name _not_ the local part. Using puny code on the local part will semantically create a different email and at last theoretically a mail provider might support both the puny code and normal version as two different mail addreses and as such using punicode there would potentially open up a different vulnarability. Now that I think about it as far as I remember the local part of mail is actually not defined as cases insensitive , through all? mail programs treat it as such. The important part her is to always use data from your database for any security relevant parts.

leowoo91 7y ago

Just checked gmaıl IDN as xn--gmal-nza.com which is registered by: 2019-02-27

hashStuff 7y ago

Wouldn't having the email hashed prevent this? Not sure why email addresses are stored plain text still. Especially if they are stored in databases.

sophiebits 7y ago
Emails are used for, among other things, sending email to a user. Which requires the application know the email address.

hashStuff 7y ago

Would this not have been solved if the email addresses were stored as hashes? Besides this it would be an extra layer of security in the even of a breach. Why addresses aren't stored as hashes seems silly. Especially when stored in true databases that can be queried quickly.

Keverw 7y ago

Reminds me of a issue Spotify had too with unicodes and usernames. https://labs.spotify.com/2013/06/18/creative-usernames/ https://labs.spotify.com/2013/06/18/creative-usernames/ I had someone tell me that programming isn't real work before, and this is yet another example of all the small little details going into building things that most people don't really think about day to day. I haven't had to work with login code in a while, but might at some point. I know some systems only allow alphanumeric usernames, but sounds like can't force people to have alphanumeric emails... Well I guess you could but might upset someone. I know there's normalizing functions though like NFD, NFC, NFKD or NFKC that might work for usernames but not sure what's really recommended. Then also brute forcing attempts to try to mitigate attacks and other considerations to make also when building out an account system. Then if your company is large enough to provide phone support, not sure how you'd tell the support person which specific emoji you used in your username.

keyP 7y ago
> I had someone tell me that programming isn't real work before Haha, I don't even understand what metrics the person was using to consider something "work". A pilot sits throughout the flight but I think it's fair to say their working...
- Keverw 7y ago
  Yeah. I guess this person thinks sitting at a computer all day isn't real work. Real work would be working in a factory all your life breaking your back. Then I also think some people think making websites and coding is the same as using Word or Powerpoint. I guess they just don't really understand tech, probably a lot of people in the rust belt. Probably why they're driving young people away.

inimino 7y ago

Domains are not case sensitive, but email local parts are! There is no reason whatsoever to do case normalization on local parts of emails on any domain you do not own, as this is strictly incorrect and could lead to a totally different address that also exists (as happened here). Of course, email providers are free to do whatever case folding or normalization they want, in which case the security burden of avoiding collisions is on the provider. If someone's email provider maps different case variants to the same mailbox, there's still no need whatsoever to do anything to the address, as the user will get it delivered to them regardless. If the provider doesn't do case folding, they will have to enter their local part case sensitively, but that's exactly the same as for any other use of their email address. I can only imagine how this vulnerability came to be. Unicode is not to blame here. If security-critical password reset code was not audited carefully enough to catch a mistake like this, one wonders what other errors might remain.

evilotto 7y ago

That's the same glyph that got a guy killed. https://gizmodo.com/a-cellphones-missing-dot-kills-two-people-puts-three-m-382026 https://gizmodo.com/a-cellphones-missing-dot-kills-two-peopl...

mrmcd 7y ago
My girlfriend is from Turkey, and I shared this story with her a while back (I discovered it while also researching some Unicode collision issues.) She said that while it's an entertaining story, there's almost certainly a bit of sensationalism and exaggeration from the Turkish press combined with credulity by English language journalists when re-reporting it. Basically, the supposed texts wouldn't have made sense in terms of grammar and syntax with a straightforward dotless/dotted I swap, and would've been obvious to someone fluent in Turkish what happened. This is would've been especially true if you had had this cell phone for any amount of time and communicating in Turkish and it had been routinely swapping Is. More likely is a bunch of young and/or not too bright people were looking for a reason to get into a violent confrontation. Then the muckraking Turkish press had a sensationalist murder-suicide lovers quarrel story, and as a bonus a nationalistic "see how cell phone companies don't respect our culture" angle as the cherry on top. However, you should still ALWAYS be careful when converting between character sets and be locality aware when manipulating strings. Practice string safety. ;)

qwerty456127 7y ago

> 'ß'.toLowerCase() // 'ss' Why? First of all ß is already lowercase, why should toLowerCase() change it? It also is a normal letter having both uppercase (ẞ) and lowercase (ß) forms so converting between the cases can be made be trivial and quirk-free. Arguably the most common word you will encounter ß in is Straße (street) where it already is lowercase - will "Straße".toLowerCase() turn it into "strasse"? WTF? "Straße".lower() returns "straße" in Python which seems reasonable (nevertheless "Straße".upper() actually returns "STRASSE" ignoring the existence of the uppercase ẞ (U+1E9E)). Why should it behave different in JavaScript? (Because JavaScript is different, I know, just a rhetorical question)

enlyth 7y ago
I must be missing something because I tried three browsers and all their JavaScript engines evaluate 'ß'.toLowerCase() as 'ß', not 'ss'.
tomcooks 7y ago
"We are living in America, Amerika ist wunderbar"
thaumasiotes 7y ago
> (nevertheless "Straße".upper() actually returns "STRASSE" ignoring the existence of the uppercase ẞ (U+1E9E)) Python probably predates the addition of a notional capital ß glyph in 2017. SS is the capitalization of ß that you'd expect if you were thinking of your data as a string rather than a collection of font elements.

ptah 7y ago

> Have we convinced you that Unicode is Awesome? this is sarcasm right?

MattConfluence 7y ago

The linked post is titled "Hacking GitHub with Unicode's dotless 'i'.", but this submission is in title case "Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'". I think this is a bad title change, because an uppercase I is supposed to be dotless, whereas the lowercase i used by the author's title is not.

reportgunner 7y ago

I haven't finished reading the article, but isn't the problem here that they are sending the e-mail to the address provided by the user rather than sending it to the e-mail stored in the database ? I fail to see any added value in sending a reset link to an e-mail entered by the user (while that e-mail is already in the database). Is it because the e-mails are stored hashed ?

angst 7y ago

>'ß'.toLowerCase() // 'ss' "ß" >'ß'.toLowerCase() === 'SS'.toLowerCase() // true false >// Note the Turkish dotless i >'John@Gıthub.com'.toUpperCase() === 'John@Github.com'.toUpperCase() true Chrome 79.0.3945.79 (Windows, 64bit) seems to differ from the proposed results for the first two statements. If the proposed results are what should indeed happen by unicode standard then I wonder chrome is not fully implementing them?

lamby 7y ago

Django has announced their security releases to match. [0] [0] https://www.djangoproject.com/weblog/2019/dec/18/security-releases/ https://www.djangoproject.com/weblog/2019/dec/18/security-re...