5 ms·

> are josh@gmail.com and jоsh@gmail.com the same user? I like to think I have a keen eye and pay attention to subtle details. I don't think I see a difference

> are josh@gmail.com and jоsh@gmail.com the same user? I like to think I have a keen eye and pay attention to subtle details. I don't think I see a difference between the two addresses. Your message seems to state they're different though. Did HN normalize the differences? Did my browser? In a blaze of stupidity, I decided to to copy and paste into my terminal and pipe it through a hex dump [0]. * The character difference doesn't show up in the viewport in Firefox; of course, it's rendered. * The character difference doesn't show up in the HTML editor; of course, it's showing the "raw text" and the "raw text" is valid unicode. * It doesn't show up in the browser's network inspection of the response payload. That's the scariest part in Firefox IMO. * It doesn't show up in my Terminal either. Why shouldn't it? We've fought long and hard for terminals to support Unicode. Of course, then there's the fact that I decided to copy and paste from a browser into my terminal even though I already knew I shouldn't [1]. What else could be hidden in unicode? An entire bash script starting with `sudo`, perhaps? Websites can already inject their shitware into the clipboard with clipboard events [1]. [0]: https://knightoftheinter.net/img/hacker%20news%20id%2023774300.png https://knightoftheinter.net/img/hacker%20news%20id%20237743... [1]: https://security.stackexchange.com/q/39118/47800 https://security.stackexchange.com/q/39118/47800

FreeFull 6y ago
The second email address uses the Cyrillic small letter O, which renders the same as the Latin small O in almost all fonts. You can see this in your hex dump, too: Instead of a single o, your hex dump shows two bytes for the second character in the second email address.
- inetknght 6y ago
  Yup! I was surprised they're rendered the same. The hex dump didn't lie even though everything until then did.
  - hombre_fatal 6y ago
    They aren't "lying", it's the same glyph in many fonts. That you think homoglyphs would somehow look different in any of those steps of your post is peculiar to me. l and I are the same in some fonts and they are in the ascii set.
    - microcolonel 6y ago
      I think part of the mistake was separating them to begin with. Far as I can tell, they are literally the same character, with the same origin.
      - TheDong 6y ago
        Unicode, in some cases, did avoid separating characters out, and that was also a mistake that damaged existing alphabets irreparably. See the Han unification [0] effort. There are characters in Japanese and Chinese which are similar, but written differently in each... And they ended up using the same codepoint in unicode for them and relying on different fonts. So now I can't easily quote a japanese sentence in a chinese book without having to use two different fonts, which seems quite silly. Worse yet, there were many common glyphs in use (especially for names) that were unified out of existence. There are literally people who couldn't type their names in unicode. There are a lot of works of text that can't be faithfully OCRd due to not having certain character variants that were unified away. Okay, so why wasn't the cyrillic alphabet unified with the latin one even if the japanese and chinese ones were? Clearly Han unification did much more damage than cyrillic unification would have. Well, the answer is sorta politics. ISO 8859 is what came before Unicode, as far as the unicode consortium is concerned. Since ISO 8859 encoded latin and cyrillic separately, that got carried over for "compatibility". Because the unicode consortium and ISO 8859 were both more western-centric, and CJK users had already dealt with things in a way that was standardized only over there, not in any western ISO standard, of course the unicode consortium would honor the existing ISO standard and ignore the CJK standards. [0]: https://en.wikipedia.org/wiki/Han_unification https://en.wikipedia.org/wiki/Han_unification
        microcolonel 6y ago
aspenmayer 6y ago
For me the first email address has the initial lowercase j seemingly normal; the second email address has the initial lowercase character that appears as j, except the below-the-line curly part doesn’t extend as far, and doesn’t upturn at all, unlike the first email’s j. This is on iOS 13. Curiously, if I copy and paste, both j’s look the same; they are formatted when pasted the same as the first initial j, which to me seems normal.
- 65536 6y ago
  I’m on iOS too. I see why you might think there are two different j there but what’s actually going on is that since the quoted text is in italics, and the text is wrapped on our iPhones so that the second italic j is on the next line of text, part of it is outside of the “bounding box” of the text of sorts, and is clipped off in rendering.
  - aspenmayer 6y ago
    I appreciate your corrections and elucidations of this typographical mystery.
oefrha 6y ago
pbpaste | xxd Or your platform equivalent of pbpaste. If this isn’t totally safe and could have side effects, your platform’s clipboard implementation is problematic.
- inetknght 6y ago
  It would be nice if the terminal would suggest a tool like that instead of just warning that it's unsafe to paste.
  - oefrha 6y ago
    Turn on bracketed paste mode everywhere. Doesn’t help with homograph attacks (e.g. an IDN homograph attack tricking you into downloading something from the wrong domain), but at least you’re totally safe until you press enter to execute. https://cirw.in/blog/bracketed-paste https://cirw.in/blog/bracketed-paste
jcrites 6y ago
The local-part of email addresses, which is the part before the @ sign, may only contain ASCII characters, so the other variant is not a valid address. https://en.wikipedia.org/wiki/Email_address#Local-part https://en.wikipedia.org/wiki/Email_address#Local-part You can probably include Unicode in the "Friendly From" header which is what is shown in most email clients though.
- akersten 6y ago
  It's somewhat comforting that that's the case. For argument's sake, we can pretend instead that the 'a' in gmail.com is the Cyrillic 'a' for one of the addresses.

FreeFull 6y ago

The second email address uses the Cyrillic small letter O, which renders the same as the Latin small O in almost all fonts. You can see this in your hex dump, too: Instead of a single o, your hex dump shows two bytes for the second character in the second email address.

inetknght 6y ago
Yup! I was surprised they're rendered the same. The hex dump didn't lie even though everything until then did.
- hombre_fatal 6y ago
  They aren't "lying", it's the same glyph in many fonts. That you think homoglyphs would somehow look different in any of those steps of your post is peculiar to me. l and I are the same in some fonts and they are in the ascii set.
  - microcolonel 6y ago
    I think part of the mistake was separating them to begin with. Far as I can tell, they are literally the same character, with the same origin.
    - TheDong 6y ago
      Unicode, in some cases, did avoid separating characters out, and that was also a mistake that damaged existing alphabets irreparably. See the Han unification [0] effort. There are characters in Japanese and Chinese which are similar, but written differently in each... And they ended up using the same codepoint in unicode for them and relying on different fonts. So now I can't easily quote a japanese sentence in a chinese book without having to use two different fonts, which seems quite silly. Worse yet, there were many common glyphs in use (especially for names) that were unified out of existence. There are literally people who couldn't type their names in unicode. There are a lot of works of text that can't be faithfully OCRd due to not having certain character variants that were unified away. Okay, so why wasn't the cyrillic alphabet unified with the latin one even if the japanese and chinese ones were? Clearly Han unification did much more damage than cyrillic unification would have. Well, the answer is sorta politics. ISO 8859 is what came before Unicode, as far as the unicode consortium is concerned. Since ISO 8859 encoded latin and cyrillic separately, that got carried over for "compatibility". Because the unicode consortium and ISO 8859 were both more western-centric, and CJK users had already dealt with things in a way that was standardized only over there, not in any western ISO standard, of course the unicode consortium would honor the existing ISO standard and ignore the CJK standards. [0]: https://en.wikipedia.org/wiki/Han_unification https://en.wikipedia.org/wiki/Han_unification
      - microcolonel 6y ago

aspenmayer 6y ago

For me the first email address has the initial lowercase j seemingly normal; the second email address has the initial lowercase character that appears as j, except the below-the-line curly part doesn’t extend as far, and doesn’t upturn at all, unlike the first email’s j. This is on iOS 13. Curiously, if I copy and paste, both j’s look the same; they are formatted when pasted the same as the first initial j, which to me seems normal.

65536 6y ago
I’m on iOS too. I see why you might think there are two different j there but what’s actually going on is that since the quoted text is in italics, and the text is wrapped on our iPhones so that the second italic j is on the next line of text, part of it is outside of the “bounding box” of the text of sorts, and is clipped off in rendering.
- aspenmayer 6y ago
  I appreciate your corrections and elucidations of this typographical mystery.

oefrha 6y ago

pbpaste | xxd Or your platform equivalent of pbpaste. If this isn’t totally safe and could have side effects, your platform’s clipboard implementation is problematic.

inetknght 6y ago
It would be nice if the terminal would suggest a tool like that instead of just warning that it's unsafe to paste.
- oefrha 6y ago
  Turn on bracketed paste mode everywhere. Doesn’t help with homograph attacks (e.g. an IDN homograph attack tricking you into downloading something from the wrong domain), but at least you’re totally safe until you press enter to execute. https://cirw.in/blog/bracketed-paste https://cirw.in/blog/bracketed-paste

jcrites 6y ago

The local-part of email addresses, which is the part before the @ sign, may only contain ASCII characters, so the other variant is not a valid address. https://en.wikipedia.org/wiki/Email_address#Local-part https://en.wikipedia.org/wiki/Email_address#Local-part You can probably include Unicode in the "Friendly From" header which is what is shown in most email clients though.

akersten 6y ago
It's somewhat comforting that that's the case. For argument's sake, we can pretend instead that the 'a' in gmail.com is the Cyrillic 'a' for one of the addresses.