10 ms·
IDN is crazy
- p-e-w 4y agoYeah, IDNs should have never happened. Homograph attacks alone are reason enough for IDNs to be a terrible idea, and they cannot be fully prevented even with special rules, because different fonts might render distinct characters similarly that are not normally considered "homographs". I feel IDNs are part of a larger trend of "Unicode should be everywhere", without any consideration for whether this is actually a good idea. Programming languages are guilty of this too, with many allowing almost any Unicode codepoint in identifiers. This has created an entirely new class of security vulnerabilities that are serious enough that VSCode now flags such codepoints by default if they look similar to certain ASCII characters. Basic ASCII Latin characters are the only characters that can be entered using almost any input device in existence without additional tools or configuration. This makes them a universal baseline, and software should carefully consider whether deviating from that baseline is really worth the security vulnerabilities and incompatibilities it may introduce, just so people can write their source code comments in Vedic Sanskrit, for a programming language where every keyword and almost every function name is in English.
- smaudet 4y agoI will probably write a character set checker for ascii tomorrow, I agree unicode is pretty much a security vulnerability anywhere but the frontend (for non copied display). Heck, it might even be a reasonable idea to bake anti-unicode into the system (kernel) and require elevated access to process it at all in a sandbox...
- Kostic 4y agoThis is a very anglo-centric view. As someone who belongs to a small nation, I'm very happy and grateful that I'm able to use my language on the Internet as much as possible.
- p-e-w 4y agoI'm not against Unicode in general. Text content on the Internet most definitely should support the full range of Unicode characters. But that doesn't mean domain names should, or identifiers in programming languages. Those are critical, fairly low-level technologies, and allowing all of Unicode raises significant security concerns. I strongly believe those concerns outweigh the usefulness of non-Latin characters in those contexts. The accusation of anglocentrism is unwarranted here. Every mainstream programming language ever created has all of its keywords based on the English language, including languages like Ruby and Python whose creators aren't native English speakers. This isn't anglocentric, it's a best-effort attempt to make the language as universal as possible in the world we live in. Internationalization at this level of technology simply leads to fragmentation. Domain names and programming languages are very different from user-focused free-form web content.
- Kostic 4y agoI'm not debating Unicode in programming languages but in domains. Also, I disagree that domain names aren't user focused. It's the first thing you learn as a regular web user (at least that used to be the case before, now it's usually the app name).
- mongol 4y agoOn the contrary, the accusation is warrranted. Domain names are printed on billboards, in newspapers, shown in TV ads and a myriad other examples outside the programming domain. I have not heard anyone from a non-English speaking country argue your viewpoint.
- zinekeller 4y agoSincere question: if Russians have first made the modern computer (and therefore the de facto language of computing is Russian and Cyrillic), are you okay with only using Cyrillic as the sole script?
- p-e-w 4y ago> if Russians have first made the modern computer (and therefore the de facto language of computing is Russian and Cyrillic) The first part of this sentence does not imply the second. The world's first programmable computer was in fact created by a German. That doesn't make German the language of computing. > are you okay with only using Cyrillic as the sole script No, because Russian/Cyrillic lacks the properties that make English/Latin so useful as a universal language/script: * Almost all languages have widely used, standardized transliteration systems into a Latin script. That's not true for Cyrillic. * English is far more widely spoken and read than any language written in Cyrillic. * The subset of the Latin script used by English is essentially the common denominator of all languages written in Latin. That's an extremely important property and immediately predisposes English for the role it has today. There are objective arguments for English as a universal language for computing. This isn't just "cultural imperialism" or something. English is not my first language, but if I try to imagine my first language being used in its place, I immediately see dozens of serious issues that would arise, all of which are non-issues with English.
- vore 4y agoI think you are doing some backwards reasoning based on the prevalence of English in computing to justify why we would only need the English alphabet. > * Almost all languages have widely used, standardized transliteration systems into a Latin script. That's not true for Cyrillic. There may be standard transliteration systems for non-English alphabet languages, but there are often multiple standard transliteration systems (e.g. Russian, Arabic). Also, Chinese transliteration is pretty gobbledygook without diacritics that aren't present in the English alphabet (and kind of gobbledygook even with diacritics). > * English is far more widely spoken and read than any language written in Cyrillic. What if the de facto language of computing was Mandarin instead? > * The subset of the Latin script used by English is essentially the common denominator of all languages written in Latin. That's an extremely important property and immediately predisposes English for the role it has today. The Latin script used by English is exactly all the letters you need to write English. Many other languages written in Latin script either have more (e.g. Spanish, Norwegian), or fewer (e.g. Italian, Serbian), or have different rules for certain things (e.g. Turkish).
- bawolff 4y agoPersonally i think the unicode code point in identifiers/source code drama is silly. The attack is someone you don't trust modifies your source code, using unicode to make the modifications more subtle. What type of threat model is that? There's tons of ways for a malicious party to sabotage source code. Most are probably more subtle than unicode hacks. It is like complaining about how the person you gave a key to your house to might rob you. I do think that in a code editor already doing syntax highlighting, individual tokens should be bidi isolated so you can't have RLO and friends affecting too much, but beyond that i really think this whole issue is way overhyped.
- josephcsible 4y ago> There's tons of ways for a malicious party to sabotage source code. If the malicious party has unfettered write access to your source code, then sure. But if all changes get reviewed by a maintainer before being merged, then there's way fewer ways, and Unicode attacks are a really large portion of them.
- TacticalCoder 4y ago> There's tons of ways for a malicious party to sabotage source code. Most are probably more subtle than unicode hacks. It is like complaining about how the person you gave a key to your house to might rob you. That's not at all how the open-source world works. Many unknown (to the project maintainer) people can create pull requests and try to sneak in a backdoor through, say, an homograph attack. if ( id == "root" && some_other_condition ) Pull requests comes in: if ( id == "rοοt" && some_other_condition && some_even_better_condition ) I totally vet that! Beautiful added security measure, that "some_even_better_condition" is the nuts, nothing can go wrong: it's an added protection! I merge that ASAP and release! Except your diff tool missed that on that same line "root" was changed to "rοοt" (and the attacker created an account named "rοοt" on your website). Game over. The point is not to bitch about my pseudo-code or how account creation do work: the point is that it's easy to miss an homograph attack during a review.
- deleted 4y ago[deleted]
- zamadatix 4y agoHomograph attacks are still a problem without unicode. The entire concept of "it looks like the right text to the user" is just a horrible security concept out of the gate. You want warnings on nearly identical looking tokens regardless the reason, even if it's not security related. As far as input devices have you used an IME hands on before? Non-latin characters aren't really any more "additional tools or configuration" than when during install you select "English" as your language. Others may select something like "中文(中华人民共和国)" and now without additional tools or configuration they are entering via pinyin instead of qwerty even though it's the same physical keyboard symbols being pressed. These IMEs don't always have a 1:1 latin character to unicode character mapping you can rely on either e.g. for the prior example on modern Windows the same set of latin characters can result in different outputs in a similar way to how english autocompletion suggestions on a mobile device work.
- thaumasiotes 4y ago> Others may select something like "中文(中华人民共和国)" and now without additional tools or configuration they are entering via pinyin instead of qwerty even though it's the same physical keyboard symbols being pressed. That doesn't work; without additional configuration you'll just end up entering the same ascii that matches the keys you press. You have to enable the IME. (In the example you appear to be suggesting, you do that by hitting shift.) You can't be in Chinese input mode all the time, because that would make it impossible to type non-Chinese text.
- zamadatix 4y agoShift will switch modes but whether you need to shift to get out of latin or vice versa depends on the context. A legacy app will almost always default to "latin, shift to pinyin" while a modern app will almost always default to "pinyin, shift to latin" - exceptions for when the app is self-aware the other input makes more sense in the context. Examples of each respectively (on the current Windows 11 at least, not sure about others) is File Explorer and the Start Menu. Regardless I don't consider this additional configuration anyways in the same way I don't consider holding shift to get capitals changing your input configuration but maybe that's just me.
- klabb3 4y agoThe problem is very real, and Unicode certainly made it worse, but you can have the same issue with ascii. Capital I and l looks the same in many fonts and ad-hoc mitigations have been made long before computers (eg license plates). And you’d be surprised how few non-techies know subdomain hierarchy rules (facebook.com.foo.bar). Add stress, dyslexia, poor eye sight etc on top of those and you can see how this becomes a real issue. Humans simply aren’t very good at string equality. As for mitigations, in ascii only you could color code lowercase/uppercase/numbers/symbols differently I guess, enforce font that renders glyphs different enough (probably monospace), highlight the domain name, etc. But even these solutions only go so far, and doesn’t naturally translate to all of Unicode or all situations where identifiers are needed.
- lifthrasiir 4y agoI actually have some firsthand experience with the IDN governance (long story short, I and other people opposed a certain kind of expansion in allowed Korean domain names [1]), and I was once told that IDN basically happened because the PRC government threatened that they will make their own DNS equivalent otherwise. I don't know how much of this is true---while who said this is supposed to know what's going on, I don't know if one actually knows that---but it did sound plausible given the haste nature of IDN and subsequent measures like LGRs. [1] https://no-hanja-domain.github.io/ https://no-hanja-domain.github.io/ (in Korean)
- Avamander 4y agoThis is such an American(TM) view it's incredible. People have the right to have computers work in their native language and alphabet, your input method be damned.
- burnished 4y agoHow do you reckon that’s a right? Seems more in the realm of “reasonable desire”. And if you want to work towards fulfilling that desire then great and more power to you, seems like it’s got some big challenges.
- Avamander 4y agoBecause people's computers belong to them, they have the right to use them in their own native language and character set. If we look at the problem from an even larger distance, it's people's right to their culture or self-expression. If these are not reasonable rights people should have then what? In the end the anglosphere should never be able to dictate the usable charset on the entire internet. That'd be utter nonsense.
- burnished 4y agoI feel like your moral outrage has sort of blinded you, did you read TFA? Talking about it in terms of rights is silly because 1) even if we agreed that it was a right it’s not something that’s being withheld, it is instead a problem no one has yet been able to solve and 2) in terms of rights everyone is already free to program in any language they want, though their choice of encoding may make interoperability with existing systems difficult
- Symbiote 4y agoI think this thread isn't discussing the article, but the borderline-flamebait comment by p-e-w that "IDNs should have never happened". I find that suggestion outrageous.
- 1vuio0pswjnm7 4y agoI disable Unicode at every opportunity I get to do so, because I honestly do not need it. Unicode as a "feature", especially one that adds complexity and potentially security risks, should be optional, not mandatory. IMHO. FWIW, I agree with this viewpoint 100%. Thanks for stating it so clearly.
- wasmitnetzen 4y agoI remember a radio ad (before IDN, obviously), where a company having an "ö" in their name had to repeatedly tell their customers that their domain is written "oe" instead. I'm sure this issue has cost them some real money. Domain names are nothing like programming languages.
- mongol 4y agoYes it used to happen all the time and it was butchering words right and left. For someone that has no affection to their language, that may feel ok, but to me the advantages of IDN outweighs the complexity without doubt.
- phendrenad2 4y agoWe don't need to throw the baby out with the bathwater here. We can have non-English domain names without the security risks. We just need to do a better job of indicating to the user that a domain name is written using, say, kanji and not ascii. Full unicode support, I.E. allowing mixed character sets in a domain name, should be disallowed, however, as displaying to the user "the first character is kanji, the rest is ascii" is too cumbersome.
- cryptonector 4y agoDisallowing mixed scripts in labels is not necessarily workable. Consider that in South Korea it is common to use the English "ing" gerund ending in otherwise Hangul text. That said, that would be the a very good step towards ameliorating the confusables issue in IDNs.
- cryptonector 4y agoThe biggest problem with IDN is that it allows mixed script labels, but even that it kinda has to (e.g., in South Korea it's common to use the English "ing" gerund ending to alter Korean words spelled in Hangul).
- Scoundreller 4y agoDanny? You here? Were you able to get into USA after your ESTA or whatever got denied?
- TacticalCoder 4y agoTime to revisit the classics from the great minds in our industry, who foresaw all these SNAFUs... Bruce Schneier, more than 20 years ago (a short read, I highly recommend it): Security Risks of Unicode I don’t know if anyone has considered the security implications of this. Unicode is just too complex to ever be secure. https://www.schneier.com/crypto-gram/archives/2000/0715.html https://www.schneier.com/crypto-gram/archives/2000/0715.html
- bawolff 4y agoComputers are made to do things. The most secure computer is an unpluged computer but that would be a useless computer. What is the alternative here? Not supporting non-ascii chars?
- josephcsible 4y agoNot supporting non-ASCII chars in domain names.
- dezgeg 4y agoPerhaps good trade-off would have been that non-ASCII is allowed only in country code TLDs, with these country operators limiting to only characters in their own alphabet. E.g. for Finland .fi could do fine with just allowing ASCII + ä, ö, å.
- gabcoh 4y agoSo are there any mitigations for curl users? Anyone know if it's possible to disable IDN by default, and only enable it when (if ever) its needed?
- kzrdude 4y agoDaniel needs to explain why it's enabled by default if it's so "crazy"
- josephcsible 4y agoBecause people who write come up with defaults are only human, so they sometimes make bad decisions, and this is one of those times.
- unwind 4y agoJust in case anyone was losing sleep over the first example, "räksmörgås" in Swedish means "shrimp sandwich". It's a well-known semi-posh dish [1] (and yes it's an open sandwich since in Sweden that is the default), as well as a fun word since it contains all of our three national characters (åäö) at the same time, thus popular among programmers when dealing with i18n etc. [1]: https://sv.wikipedia.org/wiki/R%C3%A4ksm%C3%B6rg%C3%A5s https://sv.wikipedia.org/wiki/R%C3%A4ksm%C3%B6rg%C3%A5s
- slim 4y agolooks like a salad to me (testing to see if this is a flame war vector)
- sandos 4y agoYeah, those pictures sure make it look like one. But there is a slice of bread underneath there, otherwise it is a salad! Also, no I don't this is a good way to provoke a flame war with Swedes. I can barely register an emotion after seeing this.
- teddyh 4y agoHe does not mention that domain registries will not allow such domain names to be registered.
- josephcsible 4y agoSome domain registries won't. But unless that "some" ever becomes "all", IDN attacks will still be possible.
- teddyh 4y agoThat only affects those domains who exist in those registries, on those TLDs. If my domain has a TLD managed by a registry which disallows homoglyphs, my domain is safe from impersonation.
- cryptonector 4y agoYour domain's registry's policies do not affect labels below your domain.
- teddyh 4y agoOK, so how would that work, for an attacker? If an attacker wanted to impersonate foo.com, and the .com registry does not allow homoglyphs, what could an attacker actually do?
- cryptonector 4y agoSee TFA, which shows the use of confusable homographs for forward-slash and question mark.
- teddyh 4y agoBut if, as I wrote, the registry for .com does not allow homoglyphs, then an adversary cannot register such names.
- jeroenhd 4y agoThis is the kind of mess that you end up with if you don't take internationalisation into account when you first design your standard. The impact of such an attack would've been significantly reduced if other countries' writing systems had been considered when DNS first hit the public. Punycode is a bad compromise but nothing better is available. At this point entire TLDs now rely on it, it cannot be phased out any more. Hopefully this will be a lesson next time someone invents a protocol (though I doubt it will when I read through the comments of some anglophone commenters here).
- ipython 4y agoSince dns was first defined in November 1987, well before Unicode 1.0 was ratified (1991 at the earliest), what would be your suggestion to the original implementers?
- btschaegg 4y agoTotal aside, and I have absolutely no solution for this, but: Perhaps the meta-meta-meta-problem here is that our industry is seemingly fundamentally unable to let go of flawed solutions from decades ago. Of which we have a lot -- by necessity. It's pretty much impossible to know the important flaws before something has been established and used widely enough, after which point replacing them becomes a herculean task because they have been calcified into the bedrock of our infrastructure. There are so many examples of this: - DNS (for a lot of other reasons than domain names) - IPv4 - TCP (see the motivation for QUIC) - a *lot* of incredibly obscure text encodings - IBM 3270 control codes - Sometimes even just ancient code no one understands anymore (search for the source code of Plan 9's `troff` for an example) - …and so on and so forth …and if someone wants to tackle such a situation, usually the outcome is either: - build another hack to add to the pile of hacks we won't be able to get rid of - XKCD 927 (n+1 competing standards, because the old one will live on for ever anyway) - face second system syndrome (resulting in an overcomplicated mess) - get no traction because not enough people will switch to your new and shiny solution (due to the other calcified things in the stack or new flaws with the new solution) So, my question: How should our civilization deal with this?
- 4y ago
- capitainenemo 4y agoWith regards to his unicode rendering problems that forced images of text. He should check whether his wordpress is not using a utf8mb4 configuration with mysql. Also. Perhaps domain names should be validated against the Unicode confusables list, forbidding confusable collisions. https://util.unicode.org/UnicodeJsps/confusables.jsp https://util.unicode.org/UnicodeJsps/confusables.jsp
- Avamander 4y agoMany TLDs already forbid mixing charsets. Meaning it's often possible to use only cyrillic or latin, not both. Very special unicode, including emoji, are forbidden by majority.
- capitainenemo 4y ago"Many" and "Majority" is clearly not all, so there clearly still a risk. I know there's quite a few emoji domains out there. Tutorials on it pop up on HN from time to time. Also, gonna bet that there are still confusables within character set constraints. A trivial example, the unicode confusable set considers "rn" and "m" to be confusable (in certain fonts and sizes). But I bet that's even more so in the broader european set. Also it's a bit sad to be unable to, say, mix math symbols and english... It seems to me that using the confusable set would be stricter (and safer) but also more flexible too...
- Joker_vD 4y ago> The whole Unicode range is at our disposal Sadly, no. There are actually huge swaths of Unicode disallowed in IDN [0]... although not everyone is aware of this and/or implements this check. Which is a shame, I'd really like to have e.g. "🃏-vd.name" for a homepage. [0] https://www.iana.org/assignments/idna-tables-12.0.0/idna-tables-12.0.0.xhtml https://www.iana.org/assignments/idna-tables-12.0.0/idna-tab...
- josephcsible 4y agoPSA: all Firefox users should go into about:config and set network.IDN_show_punycode to true. This will completely protect you from these kind of attacks, at least in the browser.
- jgerrish 4y agoWant to hear a 1ame joke? Look closely, and you'll see it. Your solution helps. But the better solution being pushed for, which is frankly the correct one, is "out-of-band" certification of domain names. Whether it's a little padlock next to the URL, or locking down the browser completely on non-certified domains. This will of course require more infrastructure, human and technical. And, ugh, we'll get moral leakage. Your advice is good. It's helpful if you don't fall victim to feeling too secure. But, for the debate as a whole, I'm exhausted reading through everything like a lawyer. And dealing with it as a crisis instead of a meeting of minds.
- josephcsible 4y ago> Want to hear a 1ame joke? On both my computer and phone, your "l" vs "1" swap is obvious even at a quick glance. But "lame" and "lаmе" look exactly the same in almost every font.
- cryptonector 4y agoSome of this is more a complaint about Unicode than about IDN. I'm not seeing suggestions of what would have been better. For example, regarding the confusables mappings, the list simply cannot be complete as Unicode can add new homographs in the future. Of course, IDN is crazy, but much of the craziness comes from Unicode. Still, if the craziness comes from Unicode, one could be forgiven for wishing IDN had never happened and that Unicode in domainnames was not a thing.