51 ms·

I couldn't debug the code because of my name

258 points by mikasjp 5y ago

dmingod666 5y ago
The domain name to the website is all ascii..
- zamalek 5y ago
  If you use a Microsoft account to set up windows then you have no control over the local username.
  - dmingod666 5y ago
    That sucks.. always hated the idea of an online account to access your local system..
    - zamalek 5y ago
      I'm not saying it's a good idea (even though I stupidly do it), I'm merely pointing out that there are reasons that U0080+ may end up in a username; for reasons other than intentionally putting it in there. As for the benefits, which is completely off-topic, Windows Store is actually pretty awesome if you completely avoid search (and you need to do the Microsoft account thing for it AFAIK). Windows has needed a system to update 3rd-party software, to compete with Linux package managers, and the store is a really good effort (there are still annoying warts that Aur, Deb, RPM do not have). If you're willing to be a bit dumb, there is convenience.
  - moonchrome 5y ago
    This is exactly why I don't do that initially - I don't mind my account being linked - but I've been bitten by the home path bugs multiple times, I unplug my pc during setup
bagswatchesus 5y ago
Not sure how they managed to do it but they had some basic rules that they used to say "no real name can look like this, this is a fake person!" and just kicked it out. https://www.thelvbags.co/louis-vuitton-wallets-and-purses.html https://www.thelvbags.co/louis-vuitton-wallets-and-purses.ht...
supernes 5y ago
It's somewhat common to see videogames issue a patch shortly after release where they fix crashes due to non-ASCII Windows usernames or non-English locales. I'm not sure what the root cause of the confusion is, other than text strings being hard in general.
- jerf 5y ago
  It's easy to think the answer is "just UTF-8 everything" but unfortunately the long and twisty history of filesystems means that's not the correct answer, and the "correct answer" is really hard to write down quickly. If you never display the filename, the answer is to treat existing filenames as bags of bytes, but that breaks down as soon as you need to display them, or if you need to manipulate them by appending unicode to them, in which case you have to decide on an encoding. Unicode encodings tend to mangle non-Unicode values because they're specified to replace whatever they can't understand with a particular Unicode character, usually represented as a diamond with an inverted ? inside of it. There's some obscure solutions to this problem, like https://simonsapin.github.io/wtf-8/ https://simonsapin.github.io/wtf-8/ (which includes discussion of the 16 bit encodings you need for Windows), but I haven't seen broad support for them. You need a deliberately "noncompliant" encoding/decoding system that doesn't replace unknown characters with replacement characters. Fortunately, compliant systems are becoming more and more popular and available. Unfortunately, that can make file name handling harder than when you had a non-Unicode-compliant handling system for your strings.
  - account42 5y ago
    > If you never display the filename, the answer is to treat existing filenames as bags of bytes, but that breaks down as soon as you need to display them, or if you need to manipulate them by appending unicode to them, in which case you have to decide on an encoding. No you don't. On Windows you treat paths as a u16'\' an/or u16'/'-separated sequences of uint16_t. On Unix it's a '/'-separated sequence of bytes. If you want to display, you need to decode, but for display only - so errors should use replacement characters as a graceful failure. For appending you encode your string and then append the bytes. Never do you decode externally provided paths for the purpose of manipulation. > There's some obscure solutions to this problem, like https://simonsapin.github.io/wtf-8/ https://simonsapin.github.io/wtf-8/ (which includes discussion of the 16 bit encodings you need for Windows) It's relatively new, but has wide enough adoption cosidering - e.g. it's what Rust uses for Windows paths. It's also straightforward - just encode the unmatched surrogate pairs as if they were the corresponding reserved unicode characters using the normal UTF-8 algorithm.
shantnutiwari 5y ago
the bug was fixed one hour ago-- looks like HN customer service worked again
tyteen4a03 5y ago
Fun fact: If you have the exclamation mark (!) in your Windows username, Java will think it's the jar separator and `getResourceAsStream` will refuse to work. This broke many people's Minecraft installation over the years. The bug in question [0] was reported in 2001 and remains unsolved 20 years later. [0] https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4523159 https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4523159
- alisonkisk 5y ago
  The only practical fix would be to ban ! in usernames.
- yooogle 5y ago
  Classic java
- kspacewalk2 5y ago
  I find the very idea of putting an exclamation mark on one's username and not expecting eventual problems to be quite curious.
  - retrac 5y ago
    ! is actually used as a regular letter with a phonetic value to write quite a few languages in Africa, usually representing a click sound. It's even in the name of some of them: https://en.m.wikipedia.org/wiki/%C7%83Kung_languages https://en.m.wikipedia.org/wiki/%C7%83Kung_languages
  - edaemon 5y ago
    It can be used to indicate a click sound in some languages and the IPA. Probably not the most common reason to come across this bug but it's a case worth considering. https://en.wikipedia.org/wiki/Exclamation_mark#Phonetics https://en.wikipedia.org/wiki/Exclamation_mark#Phonetics
  - jmopp 5y ago
    The Nama language has a letter that is often replaced with an exclamation mark in common typography. The lead actor in the film The God's Must be Crazy was named N!xau ǂToma
jonnycomputer 5y ago
>When I found out that the bug was in the Rider itself, I reported it to technical support. I also found a similar report for PyCharm. Unfortunately, things haven’t moved forward since then. Unfortunately, typical with Jetbrains.
trinovantes 5y ago
In CS, most algorithms assume an ASCII character set. I wonder if there's any string-related algorithms that completely break (functionally or complexity wise) when given UTF-16 or UTF-8 character sets
- deathanatos 5y ago
  > In CS, most algorithms assume an ASCII character set. They most certainly do not. E.g., a Turing machine assumes an alphabet Γ which is a set of some characters and is defined no further, as any exact definition is meaningless to the theory. (I.e., the algorithm is generic over any alphabet.) The alphabet need not even be text; e.g., for a Turing machine, the set of all octets suffices. Even for something like Levenshtein distance, the only real requirement of the algorithm is that the abstract "characters" implement equality testing. For Unicode text, I'd start with graphemes, and then look for counter examples.
  - trinovantes 5y ago
    I guess they won't break correctness but I do remember many algorithms (e.g. tries) assume you have constant random access to characters which AFAIK is not possible in UTF-8
- a1369209993 5y ago
  Asymtotic complexity can't change based on the character set, since you can just reuse the same algorithm with larger opaque datums. (Exception being algorithms with O(n^8) or O(n^256) complexity, but noone uses those anyway.) A variable width encoding can cause issues in principle, but useful algorithms already have to deal with strings that have variable-length physical represention anyway (eg "yes" vs "no"), so it tends not to be a problem in practice.
askvictor 5y ago
Somewhat surprising that this is an issue with JetBrains, given that they are based in Eastern Europe, and would probably have more direct experience of these sorts of problems than US or UK based companies. OTOH maybe it's just a scale thing - bigger companies have more resources to handle these sort of cases, regardless where they're based (not that they always do...)
PrivateButts 5y ago
Similar to this, Node and NPM get very temperamental when you have a User folder with a space in it. I gave up on the community workarounds and just created a new account and copied my files over to fix it.
SergeAx 5y ago
When I first installed Windows 7 like ten years ago, I entered my Russian name in Cyrillic. When I saw that the system created a directory with exactly that name under `C:\Users\` I immediately scanned the internet for a way to rename it and done just that. I don't want to know how much mess like that in a story I thus had successfully escaped. NB: the method is still the same, it's a second (not accepted) answer here: https://superuser.com/questions/890812/how-to-rename-the-user-folder-in-windows-10 https://superuser.com/questions/890812/how-to-rename-the-use... (about ProfileImagePath registry value).
- vertis 5y ago
  This is sad though. You shouldn't have to change who you are for a computer program.
  - GoblinSlayer 5y ago
    Is "vertis" who you are? There's more to a human, than a name.
  - SergeAx 5y ago
    I have this lower ASCII handle since about 1990, I beleive. That was the time when you just can't do literally anything without one.
godmode2019 5y ago
I have a set of names I give to different providers. Advertisers always assume a name is constant and email addresses can change. I got a name saying 'Hi John I just want to xyz' I can skip this email as they used a fake name. Works better than other methods I have found.
auggierose 5y ago
Haha, that's why something like Cosmopolitan Identifiers would be a good idea: https://doi.org/10.47757/obua.cosmo-id.3 https://doi.org/10.47757/obua.cosmo-id.3
asimjalis 5y ago
This is like Kafka’s story in which the protagonist wakes up to find out he’s a (software) bug.
ddeyar 5y ago
Some years ago I used the + feature in my gmail address. e.g. myname+ycombinator@gmail.com to track down which service is giving away my email address. It happened more than once that I could not log in anymore at some point because they started to disallow the + character in email addresses. I also got phone calls from some companies complaining that i misspelled my email address because there was their company name in it.
- cgufus 5y ago
  hehe, did the same, although not with +, but using a catch-all feature of the provider. I still get a lot of spam and phishing attempts on my „dropbox@<mydomain>“ address. I faintly remember they (dropbox) had a breach some time in the past.
Ansil849 5y ago
Sometimes even "regular" ASCII surnames cause problems. When written in the Latin alphabet, my surname is one letter. I've had an amazing amount of problems with this not just due to technical limitations (like various forms marking the entry as invalid), but--much more aggravatingly--human limitations. One particularly infuriating anecdote: at a past job many years ago, the email structure was lastname@company.com. I dutifully sent the IT person in charge of creating emails my desired email. The IT person wrote back an amazingly condescending email that as per the policy, emails had to be last names, not individual letters. I then had to go find a bunch of random websites which explained single-letter names and forwarded them to the IT person. They then obliged, but did not apologize for insulting me. That is not right that I had to put up with that.
- account42 5y ago
  > One particularly infuriating anecdote: at a past job many years ago, the email structure was lastname@company.com. I dutifully sent the IT person in charge of creating emails my desired email. The IT person wrote back an amazingly condescending email that as per the policy, emails had to be last names, not individual letters. I then had to go find a bunch of random websites which explained single-letter names and forwarded them to the IT person. They then obliged, but did not apologize for insulting me. That is not right that I had to put up with that. Except single letter last names are less common than people not following policy and/or abbriviating the name. It could simply be an honest mistake and the email is just their standard response since they have other things to get to. Did you try simply pointing out that that the letter was in fact your last name instead of getting passive-agressive?
deleted 5y ago
[deleted]
Natfan 5y ago
I've also had issues putting in my full name as my username. Lots of programs do not expect spaces in the path, and I experience a lot of errors which are resolved by changing the path to not contain a space. [1]: https://github.com/microsoft/WSL/issues/2577#issuecomment-901815008 https://github.com/microsoft/WSL/issues/2577#issuecomment-90...
itsrajju 5y ago
As of 2 hours before me writing this comment, JetBrains claims to have fixed the underlying issue [0]. Maybe they saw this post? :D [0]: https://youtrack.jetbrains.com/issue/IDEA-264563 https://youtrack.jetbrains.com/issue/IDEA-264563
deepsun 5y ago
> My username contains a "ł" character and because of it, this file cannot be processed properly. What is so curious there? Some names contain all non-latin characters, and some softwares don't work with non-ASCII symbols. I just cannot understand why is it interesting.
Svoka 5y ago
Did you know that Android still won't build on Windows if you have Cyrillic letter in user name?
souptonuts 5y ago
Idk changing your stupid fucking name could be a fix too
simonblack 5y ago
Isn't this one of those "100 things Programmers don't know about People's Names" things? Like the poor, it will be with us always.
- xdfgh1112 5y ago
  I don't know, it's just a Unicode character? Not even a newer one, it's just 2 utf8 bytes. Pretty much everything should support that in 2021. When I think of 100 things I think of stuff like "some people spell their name in all lowercase and get really funny if you change it"
  - deathanatos 5y ago
    > Pretty much everything should support that in 2021. Yes, like IPv6.
    - selfhoster11 5y ago
      UTF-8 is much less to ask for than IPv6.
  - numpad0 5y ago
    Yeah so double byte characters costs extra. I don’t know, a checkbox or something default off. Always did still does. Double width costs even more.
    - horsawlarway 5y ago
      you're getting downvoted, but between tchar hiding wchar vs char... this literally could be someone toggling off the "UNICODE" checkbox in visual studio somewhere.
  - hprotagonist 5y ago
    windows probably defaults to latin-1
    - 5y ago
mikasjp 5y ago
I think the whole problem is keeping the character encoding consistent in the applications and their dependencies. Programmers often forget this because they avoid non-ASCII characters in their code.
xwdv 5y ago
What’s wrong with just writing it as Mikolaj? It’s not like it’s a kanji or something.
- needle0 5y ago
  Then there are the people whose names ARE in Kanji, thankyouverymuch. Ah, no big deal, there's only around 1.6 billion of us.
- sophacles 5y ago
  Because that's not their name?
- wbsss4412 5y ago
  So the solution is for the user to change their entire windows account name, rather than handling common characters in your code?
  - toast0 5y ago
    For a user, changing their account (probably creating a new user, since rename apparently doesn't change the directory), is something they can do. Changing all software to respect their perfectly valid name isn't something they can do. They shouldn't need to change their name, but if they do, they can ignore all the broken software and go about their day. This particular user is more capable than most, and found a workaround for this particular problem, which is good... But this is not likely to be the last of the problems.
  - dahfizz 5y ago
    Of course it would be better if all code was bug free. But that's impossible. As a user, avoiding unicode is a pretty easy way to avoid bugs like this - its the rational thing to do.
  - Jensson 5y ago
    When you have non-standard characters in your name you quickly learn to never use them in computers since even though most systems works fine, some don't. And you can't fix all the thousands of systems your name has to interact with. I even had trouble booking flight tickets since their security system couldn't parse my name, and then had to go through some special security check due to it returning errors. After that, never again. Not sure how they managed to do it but they had some basic rules that they used to say "no real name can look like this, this is a fake person!" and just kicked it out.
jasonpeacock 5y ago
And yet it's one of the simplest things to add non-ASCII chars to your tests to validate their handling. It's like not testing if your calculate application can handle negative numbers or decimals.
- nradov 5y ago
  In fact it's trivial to generate a text file of all valid Unicode code points and use that as input to unit tests.
  - yakubin 5y ago
    It may be faster to generate them on the fly. Iterating over ranges of integers is a lot faster than reading files from disk.
  - Someone 5y ago
    I would have to do research on whether the list of valid code points depends on the Unicode version. For example, can regional indicator code points (https://en.wikipedia.org/wiki/Regional_indicator_symbol https://en.wikipedia.org/wiki/Regional_indicator_symbol) appear in isolation? If not, is that different in Unicode < 6, where those code points weren’t assigned yet? Similarly, what about tags (https://en.wikipedia.org/wiki/Tags_(Unicode_block) https://en.wikipedia.org/wiki/Tags_(Unicode_block) )? Do these require an U+E007F CANCEL TAG? The 66 noncharacters certainly need consideration. http://www.unicode.org/faq/private_use.html http://www.unicode.org/faq/private_use.html says: “Because of this complicated history and confusing changes of wording in the standard over the years regarding what are now known as noncharacters, there is still considerable disagreement about their use and whether they should be considered "illegal" or "invalid" in various contexts” Edit: also, testing all code points likely is overkill and using code points in isolation likely isn’t enough. Most tests are better of with something like the big list of naughty strings (https://github.com/minimaxir/big-list-of-naughty-strings https://github.com/minimaxir/big-list-of-naughty-strings)
umvi 5y ago
Using non-ascii characters in file paths, toolchain config files, and other non-display contexts is just asking for trouble, even if it is your name...
- PeterisP 5y ago
  Using non-ascii characters in file paths, toolchain config files and other non-display contexts is something every development team should explicitly, intentionally do in order to catch such bugs. "Asking for trouble" is a key part of testing. My suggestion would be for a QA person to have their username (and root folder of the testable project) to start with a space, and be followed by an accented letter, tab-symbol, apostrophe, an emoji, followed by an unicode RTL control character and some Arabic text.
- fluxem 5y ago
  Also spaces. I spent half an hour debugging why cmake cuda build was failing.
  - munk-a 5y ago
    A lack of support for spaces at this point is unacceptable. I, personally, despise spaces in paths but on windows a whole bunch of default system paths already have spaces embedded in them in major ways... and let's not forget parens as well - thanks "Program Files (x86)"
- b112 5y ago
  This wouldn't have happened if using rust!
  - nightfly 5y ago
    Can you knock it off??? This is even more annoying that out-of-place rust evangelism
    - GoblinSlayer 5y ago
      You know he's right. Look at all the rust converts preaching their dogmas here.
f311a 5y ago
That's a pretty common problem, especially for cyrillic names. People just use ASCII names.
deleted 5y ago
[deleted]
numpad0 5y ago
Oh, it’s not a common knowledge that you should not UTF-8 in Windows username? That had been the case since 95 days. Only recently it had supposedly improved after Microsoft Account login become semi mandatory.
- progval 5y ago
  On the contrary, the first bug happens because docker-compose tries to decode the path as UTF-8, but it is not UTF-8-encoded. ("'utf-8' codec can't decode byte")
- chris_overseas 5y ago
  I don't think this bug is anything to do with Windows, rather it is due to the way the paths are handled in the IDE's codebase. Presumably the same problem exists when using these IDEs in conjunction with a path containing non-ascii characters in the Linux or macOS world.
  - account42 5y ago
    > Presumably the same problem exists when using these IDEs in conjunction with a path containing non-ascii characters in the Linux or macOS world. Why would you presume that when the problem seems to be that one tool uses the systems native 8-bit encoding while another tool expects UTF-8 - under sane systems these are the same.
  - numpad0 5y ago
    Isn't it some compilation option issue in native part? I thought it's a line on .sln or include library in a C++ source or something that has to be explicitly specified when building a Win32 binary.
    - GoblinSlayer 5y ago
      InteliJ has native part?
- Fordec 5y ago
  A lot of adults today weren't even alive in 95. Also, the assumption that people are familiar with windows vs other operating systems is becoming less and less valid. And as the world gets more globalised and remote, it's no longer to be assumed that all technical people are of a Anglo American culture.
sschueller 5y ago
Many years ago I could not access the apple developer panel because of the umlaut in my last name. It was eventually fixed but I was quite surprised that such a large company would run into such a basic issue.
- devrand 5y ago
  My last name has an apostrophe in it which Apple apparently loves to embed directly into their JavaScript unescaped. For a long time neither I nor Apple could look up AppleCare status on my stuff as they were all linked to my Apple ID. The portal would thus require me to login, but then would just show a partially rendered page as my last name was causing an JS syntax error.
  - scollet 5y ago
    A Kafkaesque situation of no escape...
  - doubled112 5y ago
    You'd think the apostrophe would be common enough they'd know it could happen, but no. I love to enter it and see what each vendor and website's backend does with it. The Staples Canada website, for example, returns it as ' (HTML escaped) A couple times I've logged in, it seems to escape a new character. I'm currently up to &amp;#39;
    - devrand 5y ago
      Haha yeah I'm fairly used to seeing HTML escaping in my name. The weirdest case I've had with that is the Six Flags mobile app. To add a season pass you need to provide your card number and last name. For the life of me I couldn't get it to validate, but I saw they showed the HTML escaped version in their e-mails to me. Turns out I had to type out "'" into their input box for my last name as that's apparently what they put in their database.
  - nneonneo 5y ago
    Hmm, it sure sounds like John <script>alert(1);</script>Doe (Bobby Tables' distant cousin) should sign up for an Apple account. An XSS attack which could target the AppleCare reps' machines could be catastrophically bad...
spicybright 5y ago
So frustrating how this still happens. It's too latin centric.
tazjin 5y ago
The amount of random encoding problems that still exist are so bizarre. I recently left a UK job after already leaving the country more than a year ago, and in their attempt to mail P45 form to my new address (in Moscow) the only bits that survived are the string "c/o" and the postal code.
pledess 5y ago
The article offers a solution of idea.system.path=${root.dir}/JetBrains/Rider/system but doesn't mention the C:\JetBrains directory permissions. Directory permissions under %LOCALAPPDATA% (the location that works for people without a Polish character) should restrict write access to one user. With the Windows default behavior, creating C:\JetBrains would inherit permissions from C:\ - and wouldn't restrict write access to one user. Maybe 99% of the time this is irrelevant (i.e., there's no realistic threat from malicious actors who control unprivileged user accounts on your own development machine). Still, it's a potential downside of the solution, and more motivation for the vendor to fix their code so that Polish characters can be used under %LOCALAPPDATA%.
- Kwpolska 5y ago
  If you are on a multi-user system, the path "C:\JetBrains" isn’t really ideal (what if other users also need Rider and have non-ASCII usernames?). That said, you can easily change file permissions on Windows if the default ones don’t work for you.
m_kos 5y ago
Isn't it bizarre that we have self-driving cars, the ISS, and phones with 50 megapixel cameras but still struggle with character encoding?
- zakius 5y ago
  for self driving cars, ISS and digital cameras everything you do is blurry in a sense, "good enough" approximation is actually good enough while character encoding and transformations have to be done perfectly and precisely and have surprisingly big number of edge cases
- tetha 5y ago
  Character encoding is in a special class of problems. Like time handling. If you pick up a halfway non-ancient framework in a somewhat common language with a somewhat non-terrible persistence like postgres, you just don't have problems. Just don't care, and it just works. But it's super easy to derail that fragile correctness with something like MySQLs utf8-ish handling, or some OS's path handling, or 'efficiency', or a user or frontend dev submitting data in a wrong encoding. And then it gets mangled. And then the user is unhappy. At that point, it becomes very hard to argue why one of the two things is wrong, and the other is not. While the user argues the other way around. Because both look correct, if you look from the right angle. And the only reason why I am right is because of some standard, while the customer is right because of money. And yes, it is very 'surprising' why our software now functions correctly for russian or greek customers.
  - ctdonath 5y ago
    That it's a special class of problems doesn't mean it shouldn't be solved by now. Time handling should be solved too; amazing that an iOS app can't get current correct GMT.
- quadrifoliate 5y ago
  It's not bizarre at all. Character encodings are a sort of language in themselves, and end up with all the problems that regular old languages have – there's a lot of variety, people can't agree on one particular solution, and there's not a lot of money in taking care of the edge cases. It would be bizarre if we were at the point where we had perfect translations for everything, but still struggled with character encodings specifically.
amarshall 5y ago
For a list of strings that often cause problems to, e.g., add to a test suite, see https://github.com/minimaxir/big-list-of-naughty-strings https://github.com/minimaxir/big-list-of-naughty-strings
- Dunedan 5y ago
  For finding bugs caused by unexpected inputs I also find property based testing very useful. For Python there is the excellent hypothesis library for doing that: https://hypothesis.readthedocs.io/en/latest/ https://hypothesis.readthedocs.io/en/latest/
- ryanianian 5y ago
  Very handy. My previous simple test-case was simply a selection from this well-known text-file which is simply a collection of somewhat uncommon unicode characters, usually used for rendering tests. https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt But this set of strings is specifically designed to cause edge-case errors. Also don't forget Spolsky's seminal "The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)". https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/ https://www.joelonsoftware.com/2003/10/08/the-absolute-minim...
- OskarS 5y ago
  An enormously useful list, I’ve used it several times, and it can often dig up some real nastiness if you haven’t been super careful. This entry, by the way, is a fantastic little easter egg in the list: https://github.com/minimaxir/big-list-of-naughty-strings/blob/db33ec7b1d5d9616a88c76394b7d0897bd0b97eb/blns.txt#L712 https://github.com/minimaxir/big-list-of-naughty-strings/blo...
  - vertis 5y ago
    No, seriously, wake up
mrweasel 5y ago
It’s a pretty good test case. Similarly we found a number of bugs in a Django application and path handling, because I happend to be using Windows for six months, while the rest of the team was on Linux and Mac.
mkotowski 5y ago
I, too, have the Ł letter in my name, and yes, it is a sick joke that so many things even in a supposedly modern systems make an assumption that the world runs on ASCII. In the case of the Windows operating system, the worst fact is that every single part of it behaves differently. Some parts display the path with a wrong encoding, but handle it correctly. A third-party app can display it correctly, but fails while trying to access any file. From what I remember, even the built-in PATH variable editor/manager goes through some arcane steps to display the letters in a wrong way, but getting them to work sometimes. I can only imagine how much more pain it is for someone using any of the less widely-used writing systems or those with more advanced features compared to ASCII (Hebrew’s RTL, Arabic scripts mid- and final forms, etcetera).
- gerdesj 5y ago
  Can Ł have an alternative representation? For example the German ß => ss. Also I think ö can be written as oe. In English we simply shake the big bag of letters, pick a few at random and then throw them at the page until a few stick.
  - q3k 5y ago
    > Can Ł have an alternative representation? Nope. Neither can ź, ć, ś, ą or ę. You can, and people do write them as z, c, s, a and e when writing in a restriced character set, but that is not 'correct' and is not a bijection, ie. „półka” and „polka” mean two different things. There's also the case of technically-same-sounding-especially-recently ż/rz and ó/u (whose replacement would let you get rid of two 'non standard' characters), but for historical reasons these are not interchangeable.
    - gerdesj 5y ago
      I do find this sort of stuff fascinating and also faintly frustrating but of course my mother tongue is (in)famous for being a bit loose at first sight. According to one of my employees (Polish) Ł sounds roughly like w as in win or water but not as in what. A quick read of this: https://en.wikipedia.org/wiki/%C5%81 https://en.wikipedia.org/wiki/%C5%81 doesn't help too much. Does enforcing Ł instead of say w cause your written language to fail in some way? I don't want to cause offense, I want to understand the causes of difference.
xlii 5y ago
Very similar problem to one described started my exodus from Google services. I also have non-latin characters in my name however I knew it was always an issue so I never used it in paths etc. At some point, long time ago, I was tasked to do some maintance with Google Cloud service (can't remember the name of the service now) which was doable only through Python CLI utility and it failed with very similar Python error. What I found out rather quickly is that utility took my name from Google+ profile, which did include those non-latin characters. No biggie - I thought and fired e-mail to support (yeah it was those times it was still that easy). Few hours passed and I received information that this won't be fixed anytime soon and the best course of action would be to change my name. Of course, support person probably meant to remove the diacriticals from my Google+ profiles, but still it left unplesant aftertaste for years to come.
- musicale 5y ago
  > the best course of action would be to change my name That's usually easier than getting a company to fix their software.
- nullspace 5y ago
  > the best course of action would be to change my name As someone who has been told this, for other reasons, I empathize. My reaction has always been - "Your system can't even handle names, you need to fix it". Edit: I wish there was a library / service that helped you handle all sorts of edge cases in names, so that you don' t have to worry about it. Just use a user-id, and set / get a name from a lib / service that can actually handle it.
  - web007 5y ago
    I believe that library / service is called UTF-8. These days everything should be stored as bare UTF-8 data (or utf8mb4 if you're MySQL) and presented without anything else. Don't parse it, don't slice-and-dice it, don't prepend or append titles or honorifics or suffixes, don't make assumptions about length or content beyond "must be > 0 as a whole" and DEFINITELY don't use it as an identifier. Treat it as a non-unique opaque token and you'll be fine greater than 99% of the time. There are people with no last name. There are people with two or three or twelve middle names. There are people with a number for a last name. There are people with a symbol for their entire name. Take what they give you and use it and be done with it.
ygra 5y ago
One way of working arrive such issues is to use subst. That way the application thinks your project directory is actually located on P:\ or something like that.
Dannymetconan 5y ago
I can very much relate to this but also have very little sympathy here. I have a special character in my name, an apostrophe, and it causes trouble regularly online and with tooling. A number of years ago I decided just to never use it when it came to anything to do with technical work be it email, logins or usernames. Unicode characters are a pain to deal with and I have suffered from it first hand trying to handle it. At the end of the day it is much easier just to not use the special characters and move on with your life rather then be battling the constant frustration. I'm sure these tools have lots of issues opening and you would be surprised at the amount of time, effort and testing it would be required to provide fully Unicode support. Most people would see it as a very small positive and not worth the effort. I find it hard to disagree.
- johnorourke 5y ago
  I can relate, mine is O'Rourke and even in 2021 I get: - websites telling me I have an invalid name - post addressed to O'Rourke, O\\\Rourke, O&Rourke - "my account" pages say "Welcome, Mr O\Rourke"
  - Dannymetconan 5y ago
    The best one I have every seen is O\apostropheRourke for a car rental in France. I have no idea how they thought that was a good idea!
- vultour 5y ago
  I'm really surprised someone technically minded thought it's a good idea to put a non ASCII character in their username. I'd never do that.
  - stubish 5y ago
    ASCII only is not appropriate in some locales, as the keyboards don't have a-z. This is why in Thailand people tend to use their mobile phone number as their password, because it can be typed on all the common keyboard layouts they will encounter. Also, with Windows 10 users will often not even choose their username. It gets generated from their given name + surname (which is a whole different issue for people without one or t'other).
david422 5y ago
There's also this article: falsehoods-programmers-believe-about-names: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... Certainly informative if you haven't seen it before. My takeaway from it was that design your system to try to accommodate as much as possible, but it would basically be impossible to accommodate them all, so aim for your target audience.
rcxdude 5y ago
Sadly there is even still software which fails to build or even fails to run when there is a space in a filename (as is super common on windows file paths, as well as autogenerated CI build folders). It's ridiculous to no end that software cannot handle paths correctly.
darkhorn 5y ago
I think it is a Java related issue. Relevant issue occurs in Jaspersoft Report. You cannot install Jaspersoft Report on Turkish Windows no matter what.
tediousdemise 5y ago
The solution to this is extremely simple: don't validate usernames, period. The rationale is from an article someone linked here ("Falsehoods Programmer's Believe About Names"): > Anything someone tells you is their name is—by definition—an appropriate identifier for them. If you try to validate by checking for profanity, knowing full well that people can have names that contain profane substrings, I have a tongue-in-check message for you—you are a fucking asshole.
anotheraccount9 5y ago
When ł visiłed his page, my browser crashed.
lukaszkups 5y ago
Ah it's so common for me that I've totally abandoned using Latin letters in my first/last name long time ago (and I recommend the same for you ;))

258 points by mikasjp 5y ago

dmingod666 5y ago

The domain name to the website is all ascii..

zamalek 5y ago
If you use a Microsoft account to set up windows then you have no control over the local username.
- dmingod666 5y ago
  That sucks.. always hated the idea of an online account to access your local system..
  - zamalek 5y ago
    I'm not saying it's a good idea (even though I stupidly do it), I'm merely pointing out that there are reasons that U0080+ may end up in a username; for reasons other than intentionally putting it in there. As for the benefits, which is completely off-topic, Windows Store is actually pretty awesome if you completely avoid search (and you need to do the Microsoft account thing for it AFAIK). Windows has needed a system to update 3rd-party software, to compete with Linux package managers, and the store is a really good effort (there are still annoying warts that Aur, Deb, RPM do not have). If you're willing to be a bit dumb, there is convenience.
- moonchrome 5y ago
  This is exactly why I don't do that initially - I don't mind my account being linked - but I've been bitten by the home path bugs multiple times, I unplug my pc during setup

bagswatchesus 5y ago

Not sure how they managed to do it but they had some basic rules that they used to say "no real name can look like this, this is a fake person!" and just kicked it out. https://www.thelvbags.co/louis-vuitton-wallets-and-purses.html https://www.thelvbags.co/louis-vuitton-wallets-and-purses.ht...

supernes 5y ago

It's somewhat common to see videogames issue a patch shortly after release where they fix crashes due to non-ASCII Windows usernames or non-English locales. I'm not sure what the root cause of the confusion is, other than text strings being hard in general.

jerf 5y ago
It's easy to think the answer is "just UTF-8 everything" but unfortunately the long and twisty history of filesystems means that's not the correct answer, and the "correct answer" is really hard to write down quickly. If you never display the filename, the answer is to treat existing filenames as bags of bytes, but that breaks down as soon as you need to display them, or if you need to manipulate them by appending unicode to them, in which case you have to decide on an encoding. Unicode encodings tend to mangle non-Unicode values because they're specified to replace whatever they can't understand with a particular Unicode character, usually represented as a diamond with an inverted ? inside of it. There's some obscure solutions to this problem, like https://simonsapin.github.io/wtf-8/ https://simonsapin.github.io/wtf-8/ (which includes discussion of the 16 bit encodings you need for Windows), but I haven't seen broad support for them. You need a deliberately "noncompliant" encoding/decoding system that doesn't replace unknown characters with replacement characters. Fortunately, compliant systems are becoming more and more popular and available. Unfortunately, that can make file name handling harder than when you had a non-Unicode-compliant handling system for your strings.
- account42 5y ago
  > If you never display the filename, the answer is to treat existing filenames as bags of bytes, but that breaks down as soon as you need to display them, or if you need to manipulate them by appending unicode to them, in which case you have to decide on an encoding. No you don't. On Windows you treat paths as a u16'\' an/or u16'/'-separated sequences of uint16_t. On Unix it's a '/'-separated sequence of bytes. If you want to display, you need to decode, but for display only - so errors should use replacement characters as a graceful failure. For appending you encode your string and then append the bytes. Never do you decode externally provided paths for the purpose of manipulation. > There's some obscure solutions to this problem, like https://simonsapin.github.io/wtf-8/ https://simonsapin.github.io/wtf-8/ (which includes discussion of the 16 bit encodings you need for Windows) It's relatively new, but has wide enough adoption cosidering - e.g. it's what Rust uses for Windows paths. It's also straightforward - just encode the unmatched surrogate pairs as if they were the corresponding reserved unicode characters using the normal UTF-8 algorithm.

shantnutiwari 5y ago

the bug was fixed one hour ago-- looks like HN customer service worked again

tyteen4a03 5y ago

Fun fact: If you have the exclamation mark (!) in your Windows username, Java will think it's the jar separator and `getResourceAsStream` will refuse to work. This broke many people's Minecraft installation over the years. The bug in question [0] was reported in 2001 and remains unsolved 20 years later. [0] https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4523159 https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4523159

alisonkisk 5y ago
The only practical fix would be to ban ! in usernames.
yooogle 5y ago
Classic java
kspacewalk2 5y ago
I find the very idea of putting an exclamation mark on one's username and not expecting eventual problems to be quite curious.
- retrac 5y ago
  ! is actually used as a regular letter with a phonetic value to write quite a few languages in Africa, usually representing a click sound. It's even in the name of some of them: https://en.m.wikipedia.org/wiki/%C7%83Kung_languages https://en.m.wikipedia.org/wiki/%C7%83Kung_languages
- edaemon 5y ago
  It can be used to indicate a click sound in some languages and the IPA. Probably not the most common reason to come across this bug but it's a case worth considering. https://en.wikipedia.org/wiki/Exclamation_mark#Phonetics https://en.wikipedia.org/wiki/Exclamation_mark#Phonetics
- jmopp 5y ago
  The Nama language has a letter that is often replaced with an exclamation mark in common typography. The lead actor in the film The God's Must be Crazy was named N!xau ǂToma

jonnycomputer 5y ago

>When I found out that the bug was in the Rider itself, I reported it to technical support. I also found a similar report for PyCharm. Unfortunately, things haven’t moved forward since then. Unfortunately, typical with Jetbrains.

trinovantes 5y ago

In CS, most algorithms assume an ASCII character set. I wonder if there's any string-related algorithms that completely break (functionally or complexity wise) when given UTF-16 or UTF-8 character sets

deathanatos 5y ago
> In CS, most algorithms assume an ASCII character set. They most certainly do not. E.g., a Turing machine assumes an alphabet Γ which is a set of some characters and is defined no further, as any exact definition is meaningless to the theory. (I.e., the algorithm is generic over any alphabet.) The alphabet need not even be text; e.g., for a Turing machine, the set of all octets suffices. Even for something like Levenshtein distance, the only real requirement of the algorithm is that the abstract "characters" implement equality testing. For Unicode text, I'd start with graphemes, and then look for counter examples.
- trinovantes 5y ago
  I guess they won't break correctness but I do remember many algorithms (e.g. tries) assume you have constant random access to characters which AFAIK is not possible in UTF-8
a1369209993 5y ago
Asymtotic complexity can't change based on the character set, since you can just reuse the same algorithm with larger opaque datums. (Exception being algorithms with O(n^8) or O(n^256) complexity, but noone uses those anyway.) A variable width encoding can cause issues in principle, but useful algorithms already have to deal with strings that have variable-length physical represention anyway (eg "yes" vs "no"), so it tends not to be a problem in practice.

askvictor 5y ago

Somewhat surprising that this is an issue with JetBrains, given that they are based in Eastern Europe, and would probably have more direct experience of these sorts of problems than US or UK based companies. OTOH maybe it's just a scale thing - bigger companies have more resources to handle these sort of cases, regardless where they're based (not that they always do...)

PrivateButts 5y ago

Similar to this, Node and NPM get very temperamental when you have a User folder with a space in it. I gave up on the community workarounds and just created a new account and copied my files over to fix it.

SergeAx 5y ago

When I first installed Windows 7 like ten years ago, I entered my Russian name in Cyrillic. When I saw that the system created a directory with exactly that name under `C:\Users\` I immediately scanned the internet for a way to rename it and done just that. I don't want to know how much mess like that in a story I thus had successfully escaped. NB: the method is still the same, it's a second (not accepted) answer here: https://superuser.com/questions/890812/how-to-rename-the-user-folder-in-windows-10 https://superuser.com/questions/890812/how-to-rename-the-use... (about ProfileImagePath registry value).

vertis 5y ago
This is sad though. You shouldn't have to change who you are for a computer program.
- GoblinSlayer 5y ago
  Is "vertis" who you are? There's more to a human, than a name.
- SergeAx 5y ago
  I have this lower ASCII handle since about 1990, I beleive. That was the time when you just can't do literally anything without one.

godmode2019 5y ago

I have a set of names I give to different providers. Advertisers always assume a name is constant and email addresses can change. I got a name saying 'Hi John I just want to xyz' I can skip this email as they used a fake name. Works better than other methods I have found.

auggierose 5y ago

Haha, that's why something like Cosmopolitan Identifiers would be a good idea: https://doi.org/10.47757/obua.cosmo-id.3 https://doi.org/10.47757/obua.cosmo-id.3

asimjalis 5y ago

This is like Kafka’s story in which the protagonist wakes up to find out he’s a (software) bug.

ddeyar 5y ago

Some years ago I used the + feature in my gmail address. e.g. myname+ycombinator@gmail.com to track down which service is giving away my email address. It happened more than once that I could not log in anymore at some point because they started to disallow the + character in email addresses. I also got phone calls from some companies complaining that i misspelled my email address because there was their company name in it.

cgufus 5y ago
hehe, did the same, although not with +, but using a catch-all feature of the provider. I still get a lot of spam and phishing attempts on my „dropbox@<mydomain>“ address. I faintly remember they (dropbox) had a breach some time in the past.

Ansil849 5y ago

Sometimes even "regular" ASCII surnames cause problems. When written in the Latin alphabet, my surname is one letter. I've had an amazing amount of problems with this not just due to technical limitations (like various forms marking the entry as invalid), but--much more aggravatingly--human limitations. One particularly infuriating anecdote: at a past job many years ago, the email structure was lastname@company.com. I dutifully sent the IT person in charge of creating emails my desired email. The IT person wrote back an amazingly condescending email that as per the policy, emails had to be last names, not individual letters. I then had to go find a bunch of random websites which explained single-letter names and forwarded them to the IT person. They then obliged, but did not apologize for insulting me. That is not right that I had to put up with that.

account42 5y ago
> One particularly infuriating anecdote: at a past job many years ago, the email structure was lastname@company.com. I dutifully sent the IT person in charge of creating emails my desired email. The IT person wrote back an amazingly condescending email that as per the policy, emails had to be last names, not individual letters. I then had to go find a bunch of random websites which explained single-letter names and forwarded them to the IT person. They then obliged, but did not apologize for insulting me. That is not right that I had to put up with that. Except single letter last names are less common than people not following policy and/or abbriviating the name. It could simply be an honest mistake and the email is just their standard response since they have other things to get to. Did you try simply pointing out that that the letter was in fact your last name instead of getting passive-agressive?

deleted 5y ago

[deleted]

Natfan 5y ago

I've also had issues putting in my full name as my username. Lots of programs do not expect spaces in the path, and I experience a lot of errors which are resolved by changing the path to not contain a space. [1]: https://github.com/microsoft/WSL/issues/2577#issuecomment-901815008 https://github.com/microsoft/WSL/issues/2577#issuecomment-90...

itsrajju 5y ago

As of 2 hours before me writing this comment, JetBrains claims to have fixed the underlying issue [0]. Maybe they saw this post? :D [0]: https://youtrack.jetbrains.com/issue/IDEA-264563 https://youtrack.jetbrains.com/issue/IDEA-264563

deepsun 5y ago

> My username contains a "ł" character and because of it, this file cannot be processed properly. What is so curious there? Some names contain all non-latin characters, and some softwares don't work with non-ASCII symbols. I just cannot understand why is it interesting.

Svoka 5y ago

Did you know that Android still won't build on Windows if you have Cyrillic letter in user name?

souptonuts 5y ago

Idk changing your stupid fucking name could be a fix too

simonblack 5y ago

Isn't this one of those "100 things Programmers don't know about People's Names" things? Like the poor, it will be with us always.

xdfgh1112 5y ago
I don't know, it's just a Unicode character? Not even a newer one, it's just 2 utf8 bytes. Pretty much everything should support that in 2021. When I think of 100 things I think of stuff like "some people spell their name in all lowercase and get really funny if you change it"
- deathanatos 5y ago
  > Pretty much everything should support that in 2021. Yes, like IPv6.
  - selfhoster11 5y ago
    UTF-8 is much less to ask for than IPv6.
- numpad0 5y ago
  Yeah so double byte characters costs extra. I don’t know, a checkbox or something default off. Always did still does. Double width costs even more.
  - horsawlarway 5y ago
    you're getting downvoted, but between tchar hiding wchar vs char... this literally could be someone toggling off the "UNICODE" checkbox in visual studio somewhere.
- hprotagonist 5y ago
  windows probably defaults to latin-1
  - 5y ago

mikasjp 5y ago

I think the whole problem is keeping the character encoding consistent in the applications and their dependencies. Programmers often forget this because they avoid non-ASCII characters in their code.

xwdv 5y ago

What’s wrong with just writing it as Mikolaj? It’s not like it’s a kanji or something.

needle0 5y ago
Then there are the people whose names ARE in Kanji, thankyouverymuch. Ah, no big deal, there's only around 1.6 billion of us.
sophacles 5y ago
Because that's not their name?
wbsss4412 5y ago
So the solution is for the user to change their entire windows account name, rather than handling common characters in your code?
- toast0 5y ago
  For a user, changing their account (probably creating a new user, since rename apparently doesn't change the directory), is something they can do. Changing all software to respect their perfectly valid name isn't something they can do. They shouldn't need to change their name, but if they do, they can ignore all the broken software and go about their day. This particular user is more capable than most, and found a workaround for this particular problem, which is good... But this is not likely to be the last of the problems.
- dahfizz 5y ago
  Of course it would be better if all code was bug free. But that's impossible. As a user, avoiding unicode is a pretty easy way to avoid bugs like this - its the rational thing to do.
- Jensson 5y ago
  When you have non-standard characters in your name you quickly learn to never use them in computers since even though most systems works fine, some don't. And you can't fix all the thousands of systems your name has to interact with. I even had trouble booking flight tickets since their security system couldn't parse my name, and then had to go through some special security check due to it returning errors. After that, never again. Not sure how they managed to do it but they had some basic rules that they used to say "no real name can look like this, this is a fake person!" and just kicked it out.

jasonpeacock 5y ago

And yet it's one of the simplest things to add non-ASCII chars to your tests to validate their handling. It's like not testing if your calculate application can handle negative numbers or decimals.

nradov 5y ago
In fact it's trivial to generate a text file of all valid Unicode code points and use that as input to unit tests.
- yakubin 5y ago
  It may be faster to generate them on the fly. Iterating over ranges of integers is a lot faster than reading files from disk.
- Someone 5y ago
  I would have to do research on whether the list of valid code points depends on the Unicode version. For example, can regional indicator code points (https://en.wikipedia.org/wiki/Regional_indicator_symbol https://en.wikipedia.org/wiki/Regional_indicator_symbol) appear in isolation? If not, is that different in Unicode < 6, where those code points weren’t assigned yet? Similarly, what about tags (https://en.wikipedia.org/wiki/Tags_(Unicode_block) https://en.wikipedia.org/wiki/Tags_(Unicode_block) )? Do these require an U+E007F CANCEL TAG? The 66 noncharacters certainly need consideration. http://www.unicode.org/faq/private_use.html http://www.unicode.org/faq/private_use.html says: “Because of this complicated history and confusing changes of wording in the standard over the years regarding what are now known as noncharacters, there is still considerable disagreement about their use and whether they should be considered "illegal" or "invalid" in various contexts” Edit: also, testing all code points likely is overkill and using code points in isolation likely isn’t enough. Most tests are better of with something like the big list of naughty strings (https://github.com/minimaxir/big-list-of-naughty-strings https://github.com/minimaxir/big-list-of-naughty-strings)

umvi 5y ago

Using non-ascii characters in file paths, toolchain config files, and other non-display contexts is just asking for trouble, even if it is your name...

PeterisP 5y ago
Using non-ascii characters in file paths, toolchain config files and other non-display contexts is something every development team should explicitly, intentionally do in order to catch such bugs. "Asking for trouble" is a key part of testing. My suggestion would be for a QA person to have their username (and root folder of the testable project) to start with a space, and be followed by an accented letter, tab-symbol, apostrophe, an emoji, followed by an unicode RTL control character and some Arabic text.
fluxem 5y ago
Also spaces. I spent half an hour debugging why cmake cuda build was failing.
- munk-a 5y ago
  A lack of support for spaces at this point is unacceptable. I, personally, despise spaces in paths but on windows a whole bunch of default system paths already have spaces embedded in them in major ways... and let's not forget parens as well - thanks "Program Files (x86)"
b112 5y ago
This wouldn't have happened if using rust!
- nightfly 5y ago
  Can you knock it off??? This is even more annoying that out-of-place rust evangelism
  - GoblinSlayer 5y ago
    You know he's right. Look at all the rust converts preaching their dogmas here.

f311a 5y ago

That's a pretty common problem, especially for cyrillic names. People just use ASCII names.

deleted 5y ago

[deleted]

numpad0 5y ago

Oh, it’s not a common knowledge that you should not UTF-8 in Windows username? That had been the case since 95 days. Only recently it had supposedly improved after Microsoft Account login become semi mandatory.

progval 5y ago
On the contrary, the first bug happens because docker-compose tries to decode the path as UTF-8, but it is not UTF-8-encoded. ("'utf-8' codec can't decode byte")
chris_overseas 5y ago
I don't think this bug is anything to do with Windows, rather it is due to the way the paths are handled in the IDE's codebase. Presumably the same problem exists when using these IDEs in conjunction with a path containing non-ascii characters in the Linux or macOS world.
- account42 5y ago
  > Presumably the same problem exists when using these IDEs in conjunction with a path containing non-ascii characters in the Linux or macOS world. Why would you presume that when the problem seems to be that one tool uses the systems native 8-bit encoding while another tool expects UTF-8 - under sane systems these are the same.
- numpad0 5y ago
  Isn't it some compilation option issue in native part? I thought it's a line on .sln or include library in a C++ source or something that has to be explicitly specified when building a Win32 binary.
  - GoblinSlayer 5y ago
    InteliJ has native part?
Fordec 5y ago
A lot of adults today weren't even alive in 95. Also, the assumption that people are familiar with windows vs other operating systems is becoming less and less valid. And as the world gets more globalised and remote, it's no longer to be assumed that all technical people are of a Anglo American culture.

sschueller 5y ago

Many years ago I could not access the apple developer panel because of the umlaut in my last name. It was eventually fixed but I was quite surprised that such a large company would run into such a basic issue.

devrand 5y ago
My last name has an apostrophe in it which Apple apparently loves to embed directly into their JavaScript unescaped. For a long time neither I nor Apple could look up AppleCare status on my stuff as they were all linked to my Apple ID. The portal would thus require me to login, but then would just show a partially rendered page as my last name was causing an JS syntax error.
- scollet 5y ago
  A Kafkaesque situation of no escape...
- doubled112 5y ago
  You'd think the apostrophe would be common enough they'd know it could happen, but no. I love to enter it and see what each vendor and website's backend does with it. The Staples Canada website, for example, returns it as ' (HTML escaped) A couple times I've logged in, it seems to escape a new character. I'm currently up to &amp;#39;
  - devrand 5y ago
    Haha yeah I'm fairly used to seeing HTML escaping in my name. The weirdest case I've had with that is the Six Flags mobile app. To add a season pass you need to provide your card number and last name. For the life of me I couldn't get it to validate, but I saw they showed the HTML escaped version in their e-mails to me. Turns out I had to type out "'" into their input box for my last name as that's apparently what they put in their database.
- nneonneo 5y ago
  Hmm, it sure sounds like John <script>alert(1);</script>Doe (Bobby Tables' distant cousin) should sign up for an Apple account. An XSS attack which could target the AppleCare reps' machines could be catastrophically bad...

spicybright 5y ago

So frustrating how this still happens. It's too latin centric.

tazjin 5y ago

The amount of random encoding problems that still exist are so bizarre. I recently left a UK job after already leaving the country more than a year ago, and in their attempt to mail P45 form to my new address (in Moscow) the only bits that survived are the string "c/o" and the postal code.

pledess 5y ago

The article offers a solution of idea.system.path=${root.dir}/JetBrains/Rider/system but doesn't mention the C:\JetBrains directory permissions. Directory permissions under %LOCALAPPDATA% (the location that works for people without a Polish character) should restrict write access to one user. With the Windows default behavior, creating C:\JetBrains would inherit permissions from C:\ - and wouldn't restrict write access to one user. Maybe 99% of the time this is irrelevant (i.e., there's no realistic threat from malicious actors who control unprivileged user accounts on your own development machine). Still, it's a potential downside of the solution, and more motivation for the vendor to fix their code so that Polish characters can be used under %LOCALAPPDATA%.

Kwpolska 5y ago
If you are on a multi-user system, the path "C:\JetBrains" isn’t really ideal (what if other users also need Rider and have non-ASCII usernames?). That said, you can easily change file permissions on Windows if the default ones don’t work for you.

m_kos 5y ago

Isn't it bizarre that we have self-driving cars, the ISS, and phones with 50 megapixel cameras but still struggle with character encoding?

zakius 5y ago
for self driving cars, ISS and digital cameras everything you do is blurry in a sense, "good enough" approximation is actually good enough while character encoding and transformations have to be done perfectly and precisely and have surprisingly big number of edge cases
tetha 5y ago
Character encoding is in a special class of problems. Like time handling. If you pick up a halfway non-ancient framework in a somewhat common language with a somewhat non-terrible persistence like postgres, you just don't have problems. Just don't care, and it just works. But it's super easy to derail that fragile correctness with something like MySQLs utf8-ish handling, or some OS's path handling, or 'efficiency', or a user or frontend dev submitting data in a wrong encoding. And then it gets mangled. And then the user is unhappy. At that point, it becomes very hard to argue why one of the two things is wrong, and the other is not. While the user argues the other way around. Because both look correct, if you look from the right angle. And the only reason why I am right is because of some standard, while the customer is right because of money. And yes, it is very 'surprising' why our software now functions correctly for russian or greek customers.
- ctdonath 5y ago
  That it's a special class of problems doesn't mean it shouldn't be solved by now. Time handling should be solved too; amazing that an iOS app can't get current correct GMT.
quadrifoliate 5y ago
It's not bizarre at all. Character encodings are a sort of language in themselves, and end up with all the problems that regular old languages have – there's a lot of variety, people can't agree on one particular solution, and there's not a lot of money in taking care of the edge cases. It would be bizarre if we were at the point where we had perfect translations for everything, but still struggled with character encodings specifically.

amarshall 5y ago

For a list of strings that often cause problems to, e.g., add to a test suite, see https://github.com/minimaxir/big-list-of-naughty-strings https://github.com/minimaxir/big-list-of-naughty-strings

Dunedan 5y ago
For finding bugs caused by unexpected inputs I also find property based testing very useful. For Python there is the excellent hypothesis library for doing that: https://hypothesis.readthedocs.io/en/latest/ https://hypothesis.readthedocs.io/en/latest/
ryanianian 5y ago
Very handy. My previous simple test-case was simply a selection from this well-known text-file which is simply a collection of somewhat uncommon unicode characters, usually used for rendering tests. https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt But this set of strings is specifically designed to cause edge-case errors. Also don't forget Spolsky's seminal "The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)". https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/ https://www.joelonsoftware.com/2003/10/08/the-absolute-minim...
OskarS 5y ago
An enormously useful list, I’ve used it several times, and it can often dig up some real nastiness if you haven’t been super careful. This entry, by the way, is a fantastic little easter egg in the list: https://github.com/minimaxir/big-list-of-naughty-strings/blob/db33ec7b1d5d9616a88c76394b7d0897bd0b97eb/blns.txt#L712 https://github.com/minimaxir/big-list-of-naughty-strings/blo...
- vertis 5y ago
  No, seriously, wake up

mrweasel 5y ago

It’s a pretty good test case. Similarly we found a number of bugs in a Django application and path handling, because I happend to be using Windows for six months, while the rest of the team was on Linux and Mac.

mkotowski 5y ago

I, too, have the Ł letter in my name, and yes, it is a sick joke that so many things even in a supposedly modern systems make an assumption that the world runs on ASCII. In the case of the Windows operating system, the worst fact is that every single part of it behaves differently. Some parts display the path with a wrong encoding, but handle it correctly. A third-party app can display it correctly, but fails while trying to access any file. From what I remember, even the built-in PATH variable editor/manager goes through some arcane steps to display the letters in a wrong way, but getting them to work sometimes. I can only imagine how much more pain it is for someone using any of the less widely-used writing systems or those with more advanced features compared to ASCII (Hebrew’s RTL, Arabic scripts mid- and final forms, etcetera).

gerdesj 5y ago
Can Ł have an alternative representation? For example the German ß => ss. Also I think ö can be written as oe. In English we simply shake the big bag of letters, pick a few at random and then throw them at the page until a few stick.
- q3k 5y ago
  > Can Ł have an alternative representation? Nope. Neither can ź, ć, ś, ą or ę. You can, and people do write them as z, c, s, a and e when writing in a restriced character set, but that is not 'correct' and is not a bijection, ie. „półka” and „polka” mean two different things. There's also the case of technically-same-sounding-especially-recently ż/rz and ó/u (whose replacement would let you get rid of two 'non standard' characters), but for historical reasons these are not interchangeable.
  - gerdesj 5y ago
    I do find this sort of stuff fascinating and also faintly frustrating but of course my mother tongue is (in)famous for being a bit loose at first sight. According to one of my employees (Polish) Ł sounds roughly like w as in win or water but not as in what. A quick read of this: https://en.wikipedia.org/wiki/%C5%81 https://en.wikipedia.org/wiki/%C5%81 doesn't help too much. Does enforcing Ł instead of say w cause your written language to fail in some way? I don't want to cause offense, I want to understand the causes of difference.

xlii 5y ago

Very similar problem to one described started my exodus from Google services. I also have non-latin characters in my name however I knew it was always an issue so I never used it in paths etc. At some point, long time ago, I was tasked to do some maintance with Google Cloud service (can't remember the name of the service now) which was doable only through Python CLI utility and it failed with very similar Python error. What I found out rather quickly is that utility took my name from Google+ profile, which did include those non-latin characters. No biggie - I thought and fired e-mail to support (yeah it was those times it was still that easy). Few hours passed and I received information that this won't be fixed anytime soon and the best course of action would be to change my name. Of course, support person probably meant to remove the diacriticals from my Google+ profiles, but still it left unplesant aftertaste for years to come.

musicale 5y ago
> the best course of action would be to change my name That's usually easier than getting a company to fix their software.
nullspace 5y ago
> the best course of action would be to change my name As someone who has been told this, for other reasons, I empathize. My reaction has always been - "Your system can't even handle names, you need to fix it". Edit: I wish there was a library / service that helped you handle all sorts of edge cases in names, so that you don' t have to worry about it. Just use a user-id, and set / get a name from a lib / service that can actually handle it.
- web007 5y ago
  I believe that library / service is called UTF-8. These days everything should be stored as bare UTF-8 data (or utf8mb4 if you're MySQL) and presented without anything else. Don't parse it, don't slice-and-dice it, don't prepend or append titles or honorifics or suffixes, don't make assumptions about length or content beyond "must be > 0 as a whole" and DEFINITELY don't use it as an identifier. Treat it as a non-unique opaque token and you'll be fine greater than 99% of the time. There are people with no last name. There are people with two or three or twelve middle names. There are people with a number for a last name. There are people with a symbol for their entire name. Take what they give you and use it and be done with it.

ygra 5y ago

One way of working arrive such issues is to use subst. That way the application thinks your project directory is actually located on P:\ or something like that.

Dannymetconan 5y ago

I can very much relate to this but also have very little sympathy here. I have a special character in my name, an apostrophe, and it causes trouble regularly online and with tooling. A number of years ago I decided just to never use it when it came to anything to do with technical work be it email, logins or usernames. Unicode characters are a pain to deal with and I have suffered from it first hand trying to handle it. At the end of the day it is much easier just to not use the special characters and move on with your life rather then be battling the constant frustration. I'm sure these tools have lots of issues opening and you would be surprised at the amount of time, effort and testing it would be required to provide fully Unicode support. Most people would see it as a very small positive and not worth the effort. I find it hard to disagree.

johnorourke 5y ago
I can relate, mine is O'Rourke and even in 2021 I get: - websites telling me I have an invalid name - post addressed to O'Rourke, O\\\Rourke, O&Rourke - "my account" pages say "Welcome, Mr O\Rourke"
- Dannymetconan 5y ago
  The best one I have every seen is O\apostropheRourke for a car rental in France. I have no idea how they thought that was a good idea!
vultour 5y ago
I'm really surprised someone technically minded thought it's a good idea to put a non ASCII character in their username. I'd never do that.
- stubish 5y ago
  ASCII only is not appropriate in some locales, as the keyboards don't have a-z. This is why in Thailand people tend to use their mobile phone number as their password, because it can be typed on all the common keyboard layouts they will encounter. Also, with Windows 10 users will often not even choose their username. It gets generated from their given name + surname (which is a whole different issue for people without one or t'other).

david422 5y ago

There's also this article: falsehoods-programmers-believe-about-names: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... Certainly informative if you haven't seen it before. My takeaway from it was that design your system to try to accommodate as much as possible, but it would basically be impossible to accommodate them all, so aim for your target audience.

rcxdude 5y ago

Sadly there is even still software which fails to build or even fails to run when there is a space in a filename (as is super common on windows file paths, as well as autogenerated CI build folders). It's ridiculous to no end that software cannot handle paths correctly.

darkhorn 5y ago

I think it is a Java related issue. Relevant issue occurs in Jaspersoft Report. You cannot install Jaspersoft Report on Turkish Windows no matter what.

tediousdemise 5y ago

The solution to this is extremely simple: don't validate usernames, period. The rationale is from an article someone linked here ("Falsehoods Programmer's Believe About Names"): > Anything someone tells you is their name is—by definition—an appropriate identifier for them. If you try to validate by checking for profanity, knowing full well that people can have names that contain profane substrings, I have a tongue-in-check message for you—you are a fucking asshole.

anotheraccount9 5y ago

When ł visiłed his page, my browser crashed.

lukaszkups 5y ago

Ah it's so common for me that I've totally abandoned using Latin letters in my first/last name long time ago (and I recommend the same for you ;))