3 ms·

Punycode is of questionable use anyway. Granted I'm mostly looking into this from a primary latin alphabet perspective, but for the various non-latin alphabet s

Punycode is of questionable use anyway. Granted I'm mostly looking into this from a primary latin alphabet perspective, but for the various non-latin alphabet sites I've browsed over the past decade all of them just stuck to regular ASCII domains. (Heck you see this even with usernames on websites that allowed Unicode to be used; most non-latin alphabet users will still stick to the Latin alphabet for usernames.) The only place punycode really gets used are spam domains in practice. Even most Cyrillic and Asian domains don't use punycode. I get the concept of punycode and it is impressive technically but for domains it's just been a massive phishing headache.

usr1106 3y ago
I fully support national languages, ASCII is not suitable for a huge part of mankind. But it's obvious that Unicode as it is is not suitable for security critical applications. Myself, speaking several European languages, I need less than 10 Unicode characters (each of them actually still 8 bit ISO 8859-15 codes). Well, need and need, most of the sites don't even register a punycode domain, but some uglified ASCII version of their name. As a practical step browsers should ask the user whether they want to allow URLs in a certain language the first time a non-ASCII character is entered. With just one or a couple of languages allowed, the attack surface would be drastically reduced for most users.
- noirscape 3y ago
  Agreed on ASCII not being suitable for a huge chunk of mankind for messaging. That said, in lieu of a better defined set of characters, it is a reliable set of characters that largely don't overlap when it comes to homograph attacks on human readable identifiers. The only single character ones in ASCII with noticable issues are 0/O and I/l/1 (this is also why on latin alphabet gift cards, you'll often see these characters omitted entirely). The other homograph attacks on ASCII are mostly font kerning dependent (and even the two I mentioned can still be made distinct from each other in fonts). I'm not dunking on Unicode here to be clear, Unicode is awesome. That said, bolting Unicode into unique identifiers humans are meant to read is a bad idea because of the homographs. Again; just look at how non-latin alphabet countries generally (don't) work with Unicode in things like usernames. > As a practical step browsers should ask the user whether they want to allow URLs in a certain language the first time a non-ASCII character is entered. This would probably help on top of the usual algorithmic blocklists that Firefox and Chrome already use (which largely rely on trying to match what sets of Unicode are used in a domain name to pick between Unicode and punycode rendering).
- bell-cot 3y ago
  > As a practical step browsers should ask the user whether... No, humans are far too well-trained in "just say yes and forget it". The browser should loudly flag any non-ASCII name (maybe there's a buried-deep option somewhere, to less-loudly flag it), or else it should do a bit of OCR and pop up a "DANGER - Look-Alike Domain Name..." warning.
  - necovek 3y ago
    This case is a clear case of "letter k with a cedilla below": this is encoded in Unicode as a composed character that can be decomposed. No OCR needed. For all of these accented cases where no mixing of scripts occurs, browsers could simply decompose to Unicode Canonical (Decomposed) Normal Form, and flag any accented letters (eg. render them in red in the URL bar).
vbezhenar 3y ago
I live in a non-english speaking country, technically we do have non-ascii domains, but they're very rare, absolute majority of websites use latin letters. Though I think there's nothing wrong with punycode with dedicated first-level domains. like кремль.рф (does not exist, but I'm too lazy to find one).
WhyNotHugo 3y ago
> Punycode is of questionable use anyway Lots of people might want to register their name, or the name of their city, etc. These all sound like valid use cases. You could say that anything beyond ascii is of questionable use, but non-English natives will always digress.
- noirscape 3y ago
  I know these usecases. In practice, people will still just romanize their name or the name of their city to the Latin alphabet anyway. The reality is that Unicode is great for communication in say, text messages but terrible for identifiers. People in non-latin countries know this; there's a reason that there's only six countries (from what I can tell) that went with punycode TLDs while there are many more countries with unique alphabets and most of those punycode TLDs see very limited use to begin with. (The most popular being the one used by the Russian Federation, which is also the only entity to forbid using non-cyrillic characters in it's domain names).
  - yorwba 3y ago
    There's quite a lot more than just six countries with punycode country-code TLDs https://en.wikipedia.org/wiki/Country_code_top-level_domain#Internationalized_ccTLDs https://en.wikipedia.org/wiki/Country_code_top-level_domain#...
- Too 3y ago
  Humans survived with [0-9] as the main identifier for their phone line for 100 years. ASCII is actually a great improvement in that regards. Unicode just introduces too much ambiguity to be used in security critical context as in a hostname.
makeitdouble 3y ago
One of the issue with ascii domains is the phonetic mapping for CJK languages. For instance right not asahi.com is taken by 朝日 (Asahi shibun, the newspaper), thus the Asahi town (旭) cannot use it. Mind you, the town could take asahi-town.co.jp or something like that, but there is other Asahi towns and places with different writings (including 旭日, 朝陽、浅緋 etc.) Wanting all of them to have some random ascii diversification is madness, and we're only talking about Japanese places, when the Chinese character space overlaps. (The question of whether these domains are actually registered is I think a chicken and egg problem, and I have a hard time imagining the above conflict space will get a nice resolution inside the ascii alphabet) Western ascii domains being ripe for scam is an issue, throwing the baby with the bath water would still be problematic.
- GoblinSlayer 3y ago
  Ambiguity exists in all languages, e.g. what mail.com refers to?
  - makeitdouble 3y ago
    There isn't as much ambiguity in the cjk writing though. It would be similar to mail.com, mel.com, meil.com, mehl.com, melle.com and all other variations that sound roughly the same getting all mapped to meɪl.com We could live in a world where all words are written phonetically, but we don't. Expanding that courtesy to non alphabetical languages would be a pragmatic and sane approach.
- necovek 3y ago
  That seems to be mostly accidental and unrelated to ASCII: there are multiple towns named "London" and by the way that DNS works, only one of them can be "london.com" or "london.info" or... While I can see an argument for expecting to find whatever brand you think most relevant at BRAND.com(mercial), first-come-first-served is as fair as we can realistically get to (with some protections against misrepresentations and squatting). Note that I am not against IDNs: I just think your argument is flawed in their support. Again, Unicode is what allows all languages to be written out, and the fact similar-looking-URL issue is more present (it's present with ASCII too) is not a slight against Unicode and IDNs: we just need to solve for it.
zestyping 3y ago
Unpopular opinion: Trying to shoehorn all of Unicode into domain names was a bad idea. We should have stuck with case-insensitive ASCII [A-Za-z0-9-].

usr1106 3y ago

I fully support national languages, ASCII is not suitable for a huge part of mankind. But it's obvious that Unicode as it is is not suitable for security critical applications. Myself, speaking several European languages, I need less than 10 Unicode characters (each of them actually still 8 bit ISO 8859-15 codes). Well, need and need, most of the sites don't even register a punycode domain, but some uglified ASCII version of their name. As a practical step browsers should ask the user whether they want to allow URLs in a certain language the first time a non-ASCII character is entered. With just one or a couple of languages allowed, the attack surface would be drastically reduced for most users.

noirscape 3y ago
Agreed on ASCII not being suitable for a huge chunk of mankind for messaging. That said, in lieu of a better defined set of characters, it is a reliable set of characters that largely don't overlap when it comes to homograph attacks on human readable identifiers. The only single character ones in ASCII with noticable issues are 0/O and I/l/1 (this is also why on latin alphabet gift cards, you'll often see these characters omitted entirely). The other homograph attacks on ASCII are mostly font kerning dependent (and even the two I mentioned can still be made distinct from each other in fonts). I'm not dunking on Unicode here to be clear, Unicode is awesome. That said, bolting Unicode into unique identifiers humans are meant to read is a bad idea because of the homographs. Again; just look at how non-latin alphabet countries generally (don't) work with Unicode in things like usernames. > As a practical step browsers should ask the user whether they want to allow URLs in a certain language the first time a non-ASCII character is entered. This would probably help on top of the usual algorithmic blocklists that Firefox and Chrome already use (which largely rely on trying to match what sets of Unicode are used in a domain name to pick between Unicode and punycode rendering).
bell-cot 3y ago
> As a practical step browsers should ask the user whether... No, humans are far too well-trained in "just say yes and forget it". The browser should loudly flag any non-ASCII name (maybe there's a buried-deep option somewhere, to less-loudly flag it), or else it should do a bit of OCR and pop up a "DANGER - Look-Alike Domain Name..." warning.
- necovek 3y ago
  This case is a clear case of "letter k with a cedilla below": this is encoded in Unicode as a composed character that can be decomposed. No OCR needed. For all of these accented cases where no mixing of scripts occurs, browsers could simply decompose to Unicode Canonical (Decomposed) Normal Form, and flag any accented letters (eg. render them in red in the URL bar).

vbezhenar 3y ago

I live in a non-english speaking country, technically we do have non-ascii domains, but they're very rare, absolute majority of websites use latin letters. Though I think there's nothing wrong with punycode with dedicated first-level domains. like кремль.рф (does not exist, but I'm too lazy to find one).

WhyNotHugo 3y ago

> Punycode is of questionable use anyway Lots of people might want to register their name, or the name of their city, etc. These all sound like valid use cases. You could say that anything beyond ascii is of questionable use, but non-English natives will always digress.

noirscape 3y ago
I know these usecases. In practice, people will still just romanize their name or the name of their city to the Latin alphabet anyway. The reality is that Unicode is great for communication in say, text messages but terrible for identifiers. People in non-latin countries know this; there's a reason that there's only six countries (from what I can tell) that went with punycode TLDs while there are many more countries with unique alphabets and most of those punycode TLDs see very limited use to begin with. (The most popular being the one used by the Russian Federation, which is also the only entity to forbid using non-cyrillic characters in it's domain names).
- yorwba 3y ago
  There's quite a lot more than just six countries with punycode country-code TLDs https://en.wikipedia.org/wiki/Country_code_top-level_domain#Internationalized_ccTLDs https://en.wikipedia.org/wiki/Country_code_top-level_domain#...
Too 3y ago
Humans survived with [0-9] as the main identifier for their phone line for 100 years. ASCII is actually a great improvement in that regards. Unicode just introduces too much ambiguity to be used in security critical context as in a hostname.

makeitdouble 3y ago

One of the issue with ascii domains is the phonetic mapping for CJK languages. For instance right not asahi.com is taken by 朝日 (Asahi shibun, the newspaper), thus the Asahi town (旭) cannot use it. Mind you, the town could take asahi-town.co.jp or something like that, but there is other Asahi towns and places with different writings (including 旭日, 朝陽、浅緋 etc.) Wanting all of them to have some random ascii diversification is madness, and we're only talking about Japanese places, when the Chinese character space overlaps. (The question of whether these domains are actually registered is I think a chicken and egg problem, and I have a hard time imagining the above conflict space will get a nice resolution inside the ascii alphabet) Western ascii domains being ripe for scam is an issue, throwing the baby with the bath water would still be problematic.

GoblinSlayer 3y ago
Ambiguity exists in all languages, e.g. what mail.com refers to?
- makeitdouble 3y ago
  There isn't as much ambiguity in the cjk writing though. It would be similar to mail.com, mel.com, meil.com, mehl.com, melle.com and all other variations that sound roughly the same getting all mapped to meɪl.com We could live in a world where all words are written phonetically, but we don't. Expanding that courtesy to non alphabetical languages would be a pragmatic and sane approach.
necovek 3y ago
That seems to be mostly accidental and unrelated to ASCII: there are multiple towns named "London" and by the way that DNS works, only one of them can be "london.com" or "london.info" or... While I can see an argument for expecting to find whatever brand you think most relevant at BRAND.com(mercial), first-come-first-served is as fair as we can realistically get to (with some protections against misrepresentations and squatting). Note that I am not against IDNs: I just think your argument is flawed in their support. Again, Unicode is what allows all languages to be written out, and the fact similar-looking-URL issue is more present (it's present with ASCII too) is not a slight against Unicode and IDNs: we just need to solve for it.

zestyping 3y ago

Unpopular opinion: Trying to shoehorn all of Unicode into domain names was a bad idea. We should have stuck with case-insensitive ASCII [A-Za-z0-9-].