24 ms·

Debian opens a can of username worms

249 points by jwilk 2y ago

rini17 2y ago
Perhaps it's time to agree upon how to Unicode in identifiers? The normalization, unprintable characters, confusing characters with same glyphs, etc. It's obviously problematic when everyone is doing it on their own.
- magicalhippo 2y ago
  As long as I can enter my Zalgo[1] username, I'm fine with your suggestion. [1]: https://en.wikipedia.org/wiki/Zalgo_text https://en.wikipedia.org/wiki/Zalgo_text
- m000 2y ago
  Good luck bringing everyone together. There's still a ton of Microsoft software that relies on the presence of the BOM [1], despite practically everyone else not using it. And bidirectional rsync between practically everything else and a Mac still requires `--iconv=utf-8,utf-8-mac` to avoid problems because of homographs. [1] https://en.wikipedia.org/wiki/Byte_order_mark https://en.wikipedia.org/wiki/Byte_order_mark
- bayindirh 2y ago
  The first bar to clear is "The Turkish Test"[0], then we can talk about Unicode. It'll smooth the rest of the process a lot. You can't guess how many workarounds I implement to make sure that a stray application doesn't get "ı" or "İ" in their naive codepaths, and start burning mid-flight (e.g.: Kodi, Pagico, some old Java programs, oh my...). [0]: https://blog.codinghorror.com/whats-wrong-with-turkey/ https://blog.codinghorror.com/whats-wrong-with-turkey/
  - beardyw 2y ago
    The date format part is ridiculous. Americans are almost unique in using mm/dd/yyyy, so an assumption of that would be plain wrong.
    - bayindirh 2y ago
      Localization libraries handle these parts well, since date is same with Europe (and generally stored as time-date objects rather than pure strings). None of the number shenanigans cause problems since these numbers are always stored as IEEE754 or other decimal formats. Money is no problem as well. However, when you go through an upper() or lower() or anything which plays with capitalization, and if that data is being fed to a hash algorithm or anything which mucks with strings, boy, oh boy... The easiest way is to sanitize these programmatic parts with forced locale of en_US or plain old "C". If the strings is not facing to the user and never localized, just force its locale. It's the only sane way.
      - oblio 2y ago
        > since date is same with Europe Do you mean MM-DD-YYYY? No, the vast majority of Europe does DD-MM-YYYY in some form or another.
        bayindirh 2y ago
        No, I mean DD-MM-YYYY. We use the same format with the vast majority of Europe.
        oblio 2y ago
        I'm confused, are you talking about the US? The US for sure does not default to DD-MM-YYYY.
        bayindirh 2y ago
        I'm talking about the Turkish Language, its locale and its peculiarities since it has letters "ı" and "İ". So, I'm talking about Turkish date format. Turkey uses DD-MM-YYYY format, like the most of the Europe.
    - a3w 2y ago
      https://xkcd.com/1179/ https://xkcd.com/1179/ I heard the US and A are moving to the hissing cat date format shown here.
- throw0101a 2y ago
  > Perhaps it's time to agree upon how to Unicode in identifiers? And then update all data structures that refer to them (like last and w/who, also NFS), as well as file formats (like cpio, tar, and pax which encodes ownership).
  - maccard 2y ago
    Yes. Those formats have had 20 years since Unicode was standardised, and things like my terminal still routinely break when given “unexpected” inputs. Practically every other application can handle it.
- layer8 2y ago
  Unicode has provided a specification for Unicode identifiers since 2005: https://www.unicode.org/reports/tr31/ https://www.unicode.org/reports/tr31/
  - rini17 2y ago
    Great! Is there a library for their validation? ICU seems to have only spoof checker for confusables.
    - rurban 2y ago
      libu8ident
      - westurner 2y ago
        rurban/libu8ident : https://github.com/rurban/libu8ident https://github.com/rurban/libu8ident : > unicode security guidelines for identifiers
    - westurner 2y ago
      ICU: International Components for Unicode: https://en.wikipedia.org/wiki/International_Components_for_Unicode https://en.wikipedia.org/wiki/International_Components_for_U... unicode-org/icu: https://github.com/unicode-org/icu https://github.com/unicode-org/icu Microsoft/ICU: https://github.com/microsoft/icu https://github.com/microsoft/icu IDN: Internationalized domain name: https://en.wikipedia.org/wiki/Internationalized_domain_name https://en.wikipedia.org/wiki/Internationalized_domain_name Punycode: https://en.wikipedia.org/wiki/Punycode https://en.wikipedia.org/wiki/Punycode IDN homograph attack: https://en.wikipedia.org/wiki/IDN_homograph_attack https://en.wikipedia.org/wiki/IDN_homograph_attack CWE-1007: Insufficient Visual Distinction of Homoglyphs Presented to User: https://cwe.mitre.org/data/definitions/1007.html https://cwe.mitre.org/data/definitions/1007.html GNU libidn/libidn2: https://gitlab.com/libidn/libidn2 https://gitlab.com/libidn/libidn2 Comparison of regular expression engines > Language features > Part 2; Unicode support: https://en.wikipedia.org/wiki/Comparison_of_regular_expression_engines#Part_2 https://en.wikipedia.org/wiki/Comparison_of_regular_expressi...
- secondcoming 2y ago
  Would punycode be suitable?
tiahura 2y ago
When you think about all the time, money and effort that have been wasted on Unicode...
- kalleboo 2y ago
  Yeah we should have all just stuck to Shift-JIS
- Joker_vD 2y ago
  Всеки трябва да използва латиница, абсолютно съм съгласен. After all, it's objectively the most perfect set of characters for any reasonable human language.
  - febusravenga 2y ago
    Random cross-language-script observation. In Bulgarian, латиница ("latin alphabet") transliterated to latin alphabet is just "latinitsa" or "latinica". In Polish "cyrillic" is "cyrylica" - basically reverse.
- pjc50 2y ago
  What's your preferred solution for representing the CJK languages?
  - tiahura 2y ago
    Computing did pretty well in the prior 50 years.
    - pjc50 2y ago
      That's not an answer. Be specific. How do you want to represent the 97k CJK characters?
      - vman81 2y ago
        I really don't want to be snarky or sarcastic, so I'll just be plain. Many people are unwilling or unable to understand a problem that doesn't affect them directly. Like - "UTF is woke" kind of people. They are out there.
card_zero 2y ago
> naming things is one of the hard things to do in computer science I've been thinking about that a lot lately. Code is text, it's arranged linearly, code has to be readable, identifiers are thus short strings that try to express short essays about the purpose of the variable or whatever it is, and then ideally there's a longer version of the essay in a comment, but not too long because that would clutter up the code as well (because it's text, arranged linearly). And we have code folding to tidy them up, for what good it does, and ideally an even longer version of the essay in documentation except nobody writes that. What if it wasn't text, and wasn't linear, and we didn't have an expectation that code should be strings of stupid over-terse names and hieroglyphic symbols? So I was thinking vaguely about investigating graphic-based programming, but it's probably worse, IDK. It could automatically assign arbitrary icons* instead of identifiers, and you could write tooltip-like comments to describe them as and when you want to, and everything could be laid out nicely with diagrams and different pages instead of like a text file. I suppose this is all merely cosmetic? The thing with the instance on code being written as strings of text feels very primitive, is all. It causes this problem. * Which doesn't solve the problem, I admit, because now you have to remember what the icons mean, but maybe that's easier?
- jstanley 2y ago
  I don't think remembering the meaning of icons is easier, because in order to think about it you have to be able to pronounce it inside your head. And code isn't just linear, it can be spread across multiple files in a directory tree, functions can can each other, etc.
  - c22 2y ago
    > in order to think about it you have to be able to pronounce it inside your head. I'm not sure this is universal.
    - vidarh 2y ago
      Indeed, some people do not even have an inner voice, the same way some of us don't "see" things in our minds eye. Neither prevents you from thinking about words or visual objects.
mmsc 2y ago
I wonder how this will affect ssh. OpenSSH recently restricted more characters for valid usernames: https://github.com/openssh/openssh-portable/commit/7ef3787c84b6b524501211b11a26c742f829af1a https://github.com/openssh/openssh-portable/commit/7ef3787c8...
- cedws 2y ago
  This is a great example of how one poor decision, or one piece of code that is too liberal cascades into an avalanche of shitty workarounds.
- throw0101a 2y ago
  It should be noted that shell metacharacters are also not allowed under POSIX: * https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_437 https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j k l m n o p q r s t u v w x y z 0 1 2 3 4 5 6 7 8 9 . _ - * https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282 https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... (Hyphen forbidden as first character.)
- linuxftw 2y ago
  I think it will be fine. Everyone will quickly learn the lesson "Use something other than ASCII letters and numbers at your own peril." Similar to people who put spaces in file names, it should be a fire-able offense.
  - deleted 2y ago
    [deleted]
  - lexicality 2y ago
    any software that can't handle spaces in filenames is broken
    - Muromec 2y ago
      All of the software is broken (including security wise) all the time anyway.
dfranke 2y ago
Allowing purely numeric usernames seems like a terrible idea to me, because it creates ambiguity between what's a username and what's a UID. It's common for tools like ls or ps to display a username when one is found and fall back to displaying a UID if it isn't, and similarly tools like chown will accept either a UID or a username and disambiguate based on whether it's numeric or not. Now suppose there's a numeric username that doesn't match its own UID, but does match some other user's UID. It doesn't take a lot of imagination to see how this would lead to vulnerabilities.
- throw0101a 2y ago
  Talk to POSIX: > A string that is used to identify a user; see also User Database. To be portable across systems conforming to POSIX.1-2017, the value is composed of characters from the portable filename character set. The <hyphen-minus> character should not be used as the first character of a portable user name. * https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_437 https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... The "portable filename character set" is defined as: A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j k l m n o p q r s t u v w x y z 0 1 2 3 4 5 6 7 8 9 . _ - * https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282 https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... So only a hyphen as the first character is forbidden. Given that you can't necessarilly control where usernames come from (e.g., LDAP lookups), properly speaking your system has to handle everything anyway, even if you don't allow local creation.
  - dfranke 2y ago
    Yes, I'm aware, and POSIX has many such bugs that make command input or output unavoidably ambiguous if certain unexpected characters are present that they didn't think to prohibit. A lot of the revisions that went into POSIX 2024 were aimed at fixing some of these, such as standardizing find -print0 and xargs -0. The fact that this one got overlooked doesn't mean it's a good idea to make the situation worse and harder for future POSIX revisions to address.
  - bluGill 2y ago
huhtenberg 2y ago
Sound like a solution in search of a problem. And a disruptive solution with unclear side effects at that.
johnisgood 2y ago
> If a keyboard input system provides the former sequence of bytes, but the username is stored in the login infrastructure using the latter sequence of [bytes], then a naive comparison will not find the user "émollier" in the system. Unicode defines in Annex 15 a few normalization forms as a way to work around this problem. But a correct use of these normalization forms still requires coordination and standardization among all programs accessing the data. ICU could work, but adds an extra dependency, there is also GNU's libunistring.
resource_waste 2y ago
This is important because Debain-family is used on many servers? Debian seems to just squander resources on things a few powerful people care about. All my servers have been Debian-based, so I can't be too hard on them, but whenever I see someone recommend a Debian-family distro as a Desktop OS, I feel like I need to call the police.
perlgeek 2y ago
Just imagine how many poorly-written shell scripts will break when we suddenly allow dollars, quotes, backticks and the likes in username. Heck, even allowing spaces sound like horror to me. On the display side, I'm sure most tools that display usernames won't make it easy to see if there are leading or trailing whitespace characters, double blanks, tabs etc in usernames. This sounds like support hell to me.
- gmuslera 2y ago
  The problem could be old scripts or systems that doesn’t handle UTF-8 (that doesn’t need to be the ones where the username was defined). I’m not sure if I.e. the Bobby tables trick could be done with characters with UTF8 representation seeing them in pure ascii.
- deleted 2y ago
  [deleted]
- Starlevel004 2y ago
  Breaking shell scripts sounds like a good idea to me. The faster they die the better the world gets.
  - Rygian 2y ago
    That's going to be a very bumpy road, even if everyone were to agree that the destination is appealing.
    - bigstrat2003 2y ago
      Yeah for better or for worse compatibility is king. I despise shell scripts, they are an absolute nightmare to work with and full of footguns. But they are so commonplace that people are not going to tolerate YOLO breaking changes.
    - raverbashing 2y ago
      Yeah I think ESH While we have more modern shells the fact that bash (or even sh) is the "common denominator" 30 yrs on is both good and awful We need a PowerShell for Linux
nmstoker 2y ago
Unfortunate ambiguous uses of the word drop throughout the otherwise excellent article
- TimK65 2y ago
  There are three uses of the word "drop," all of which are correct. The latter-day meaning of "drop" is an abomination.
  - toast0 2y ago
    I dropped X off at Y. Then X dropped off the face of the map, never to be seen again. Many words and phrases in English are self-antonyms.
- fargle 2y ago
  > The src:shadow package had dropped a Debian-specific patch, shoot, that's evil. had not noticed this. i read this as "removed", not "was released". now idk. this pseudo-definition of dropped as "released" is beyond stupid. yikes!
miohtama 2y ago
I remember useradd and adduser when learning Linux and oh boy what a confusion it was... Why not just one command
account42 2y ago
Always fun to see people poke the Unicode dragon only to be dumbstruck by its true size as it stands up in preparation of engulfing them with the fire of unintended consequences.
- beardygo 2y ago
  Indeed. As a speaker of several languages, including RTL language (they haven't even considered the problems with RTL marks etc), I say stay with ASCII for usernames, keep UTF for full names. If restricted ASCII a-z is good enough for passport names worldwide, it's good enough for usernames.
  - macbr 2y ago
    I’m confused – my name as written on my passport definitely contains non ASCII characters?
    - extraduder_ire 2y ago
      What is it in the machine-readable section at the bottom? My passport takes the apostrophe out of my name down there.
      - belorn 2y ago
        What is the point of a machine-readable name when there is a machine-readable passport number which should be unique for each issuing country? In this age I would assume that places which uses machines to read passports also are connected to international databases where the unique number is checked for validation. My country also mandated passport with chips in them for the last couple of decades, so by now there are no longer any valid passports without such chip. If I had to guess, it seems the machine-readable section is just backward compatibility for machines built during the period where people started doing machine reading of passports but had yet to started to put chips into them. (as a fun side note, smart phones can read the chip on passports and this is then used by some digital identity providers to establish identity on account creation, in combination with the phone camera).
abigail95 2y ago
if you cannot handle UTF-8 anywhere anything approaching text could be, your program is malformed and should be deprecated and removed. if you wrote code that couldn't handle bob;>/hacked in a username, you would and should be laughed at. why are we using this ancient stuff?
- knorker 2y ago
  It's not just programs. And it's not just semantics of all-numeric username. It's also whether you want usernames that you cannot type, nor possibly even render. Definitely you can't spell it to someone else. Who owns that file? Oh, it's right-to-left non breaking space smiley snowman Chinese sign for water, I love that guy!
  - abigail95 2y ago
    If people want to set up a Debian environment where people are mixing RTL and Hanzi I see no reason for that to be prohibited. Debian has opinions but I disagree that they should extend that far. If my employee Zalgo-fies everything. I don't file a bug report with Debian. I just fire them.
    - Muromec 2y ago
      >If my employee Zalgo-fies everything. I don't file a bug report with Debian. I just fire them. Which such clearly north American attitude you can as well use ASCII for everything.
  - Izkata 2y ago
    > Who owns that file? Oh, it's right-to-left non breaking space smiley snowman Chinese sign for water, I love that guy! This reminds me, around 10 years ago on the chat app we used at work, we were able to change our nicknames and I made mine start with a combining character instead of a regular character. No one could ping me, it broke that part of the UI when they tried.
  - adrian_b 2y ago
    This thread like also the parent thread is full of comments which are completely outdated, because there already exist standards for Unicode identifiers and obviously they forbid such cases. See e.g. RFC 8264. Only a restricted set of characters is permitted in identifiers, mostly letters and digits. This is enough to write any user name, without allowing "smiley snowman Chinese sign for water" or other such nonsense.
chikere232 2y ago
oh yes, let's break things to gain nothing of value
- gspr 2y ago
  Perhaps nothing of value _to you_. I'll hazard a guess that your preferred username can be expressed in a small subset of ASCII? And to hell with everyone else?
  - knorker 2y ago
    I'll hazard a guess that your preferred username can't be written by 99.99999% of the world, and would always have to be copy-pasted?
    - Ylpertnodi 2y ago
      Yeah, us foreigners, up to our usual tricks again.
      - knorker 2y ago
        By any definition of the word, I'm a foreigner. So if you meant to imply that I'm an American, you've guessed wrong.
    - gspr 2y ago
      My preferred login name, and in fact also my legal name, are ASCII safe. I've never personally had to fight computer systems built with ridiculous assumptions (in the domain of names, at least). What I'm exhibiting in this thread is called empathy for people who don't happen to land in that category.
      - knorker 2y ago
        Did you appreciate my comment's real meaning, that one can have empathy for everyone else too, who need to work with this data? Another solution is, of course, to give everyone usernames they don't want. E.g. your user name is your employee number. Or your department plus three letters from your name that happen to make your username unique. I've seen both. I also extend my empathy to those who, after choosing a username nobody can type, don't get invited to meetings, don't get messages, are never referred to as the domain expert, because how can I add "习近平" to a meeting, if I can't even search for their username? Say even "mårten" is not showing up when I search my email, because those are not the same letters as "marten". Meh, emacron knows almost as much on this topic, so I'll just invite him instead. If you're already powerful enough, then people will be forced to copy-paste "mårten". If you're not already powerful enough, then you don't get invited at all, because that doesn't scale.
codedokode 2y ago
Don't you think that it would be better to get rid of usernames in UI? They only provide unique data for fingerprinting and do almost nothing useful on a single-user system. Wouldn't it be better to simply have a default name like "primary user" or "main user" for the first user and skip one step in installation process? Also it frees you from typing a username on login for a single-user system.
- eviks 2y ago
  Single user systems can just not ask for a username if there is only one, they control the UI
knorker 2y ago
So in the future I may not be able to even type the name of another user? Admins and other users not being able to type usernames sounds very bad. And I say that as someone whose native language has more letters than English.
zvr 2y ago
Most people are too young to remember that when you typed your username in all-caps in the login prompt (because the CapsLock key was on by accident, for example), the login(8) program assumed you were in a connection that could only do 7-bit (upper case, but no lower case characters) and immediately switched the tty settings and you were then presented with a "\PASSWORD: " prompt.
- roelschroeven 2y ago
  Don't you mean 6-bit? 7-bit ASCII supports lower case characters. Or maybe there are other 7-bit character sets that don't have lower case characters and it was one of those?
  - jks 2y ago
    PETSCII? On the Commodore 64 you could press the Commodore key and Shift together to change character sets between lowercase and the graphical characters. But the Unix login thing might have been because of teletypes? https://www.columbia.edu/cu/computinghistory/teletype/ https://www.columbia.edu/cu/computinghistory/teletype/ claims that ASR 33 used 8-bit ASCII but was uppercase only - not sure if the "8-bit" claim can be true. On some Unix (and Linux) systems, you can still enter a kind of retro mode with "stty olcuc iuclc" (output lowercase to uppercase, input uppercase to lowercase) and turning on Caps Lock.
  - zvr 2y ago
    You are of course correct that 7-bit ASCII includes lower case characters. I don't think there exists "6-bit ASCII", but the original ASCII did not have lower case (the slots were empty). We're talking early '60s here. I'm not even sure it was only about ASCII. I suppose I should have written a more generic "character set" (which supports or not lower case characters) rather than "7-bit". In the cases where you could only communicate in a single case (upper), you typed the commands in the usual letters (e.g., "LS") and capital letters were designated by a preceding backslash (e.g., "ECHO \JOHN \DOE"). That's why you were seeing the "\PASSWORD: " prompt, the initial letter was capital (as it still is). Just for fun, I checked my current Debian system. The getty(8) command still supports it: -U, --detect-case Turn on support for detecting an uppercase-only terminal. This setting will detect a login name containing only capitals as indicating an uppercase-only terminal and turn on some upper-to-lower case conversions. Note that this has no support for any Unicode characters.
soneil 2y ago
This reminds me of the systemd bug where usernames starting with a digit were mishandled (#15141). It seems to me like something that "should" be relaxed, but we need to have high confidence in the entire foodchain. adduser seems like the last place it should be changed, not the first - anyone requiring "enough rope" is already served by useradd.
hwc 2y ago
My work machine uses my complete email address as a user machine (this was a done by someone in the IT department). Vim gets confused when I use the `gf` command to open a path that contains an '@' character in it.
bjourne 2y ago
Honestly, it is super brain-dead that Linux and other operating systems still have such massive problems with "special" characters. Just the other day I had to help someone who had trouble building. The cause turned out to be that they had dropped filenames with parentheses in the source directory which, apparently, confused bash which make relies on. Such trash is everywhere on Linux systems. Eventually you learn to only use [a-zA-Z0-9-_.] in names because anything else will inevitably confuse some tool or another (even capital letters can be a PITA)... I so wish someone would take it upon themselves to clean up this mess, but it's probably too much work and too many who are nay-sayers conditioned to it who don't see the need for changes.
hiccuphippo 2y ago
As someone who needs non-ascii characters to write my name: please don't. You are making things worse just to be "courteous" about something we don't care about and will actually be annoyed at if we have to find how to write a letter in the keyboard or worse case scenario, figure out how to change the layout to the correct one before I even logged in.
- jks 2y ago
  Likewise. My last name contains a non-ascii character. In ~2009 I started at a company whose admin conveniently set up an account for me on their Ubuntu server... on which no-one could then log in locally because the login manager crashed when trying to display the list of users. I logged in via ssh and changed my name to the nearest ASCII equivalent. I always feel slightly worried on sites that demand that I give my full legal name (such as the US ESTA form), and then refuse to handle it because it includes "illegal" characters.
  - ASalazarMX 2y ago
    This has happened to me with passwords containing foreign characters. The system would accept it, but further logons would be impossible. Now I always strip diacritics to be safe.
    - jks 2y ago
      A friend mentioned using control characters in passwords... like ^F and ^B, but not ^C because that's the interrupt character. Feels vaguely risky to me (does ^U empty the line? does ^W delete the last word? does your terminal emulator do some weird encoding like it does for cursor keys?) but if it works, why not?
      - jowea 2y ago
        I suspect I have run into a couple bugs because of password generators putting characters that some backend system cannot process in the password. Halfwish they just did DKWhhjwqjkwqjmHSJKHAIUHQwdmlsadkl instead.
SuperSandro2000 2y ago
They are clearly bored and want to start a year long bug hunt through half of unix
- Muromec 2y ago
  That sounds like a good kind of bored and bug hunting through half the unix sounds like fun too.
  - db48x 2y ago
    Agreed! I’m half tempted to join the hunt myself just for kicks…
kej 2y ago
I wonder if it would work to do something like the punycode system for internationalized domain names. Shell scripts could handle a name like `xn--0civ130n` just fine, and user-facing utilities could choose to convert that to :sparkle::unicorn: when appropriate. The same homograph protections would probably work, as well.
dsr_ 2y ago
I will remind everyone that there are a minimum of three identifiers here. The UID, which is an integer. Ownership resides here; it's the primary key. Can be used by programs. The username, each of which must be unique and maps to one UID -- but multiple usernames can map to the same UID. Used by humans and programs to login. The GECOS field, or "human readable name", which is only used as a display label. Some systems include a structure inside this for additional info like phone number, office number, or similar". I don't think anyone would object to UTF-8 here.
seu 2y ago
The fact that this whole discussion happens in english, partially explains why there is a discussion at all. The whole problem could have been avoided if the development of computers had been a more international effort.
- alterom 2y ago
  [flagged]
- alterom 2y ago
  >The fact that this whole discussion happens in english, partially explains why there is a discussion at all. The whole problem could have been avoided if the development of computers had been a more international effort. Your comment makes for a solid example of why DEI efforts have a very practical side to it, and actually makes solid sense in the 21st century.
seiferteric 2y ago
OMG Can't believe this, I ran into this exact thing at my last job. We discovered a security vuln in several of our services because we were accepting unsanitized usernames, but since we and doing things with them (passing them to scripts etc.) but only after passing them to useradd/usermod etc so we thought they were safe, and of course you could put in things like ";" and "&", ">" etc and do whatever you want. I discovered that debian DISABLED the username sanity checks and could not believe it. anyway I installed a patched version as well as sanitized input and other stuff to resolve the issue.
IshKebab 2y ago
> Most Debian users don't work with useradd, or groupadd, directly. Instead, Debian has long supplied its own adduser (and addgroup) utilities, originally written by founder Ian Murdock. These act as simpler front ends to useradd One of the dumbest things Debian has done.
rurban 2y ago
They are so stupid, I cannot believe! Names are identifiers, and such need to stay identifiable. There exist unicode security guidelines and rules for identifiers, they don't know about. My libu8ident library would help with that.
_yb2s 2y ago
Clearly we should open up usernames to be an unlimited size set of mixed data types: e.g. the first “character” could be a hand drawn picture of a cat, the second the entire text of the US constitution in unicode, and so on. We could then extend this flexibility to filenames, passwords, and Unix commands. Internally, this could involve replacing all text strings with folders on a filesystem where you can put any files you want in any desired order. /s
- adrian_b 2y ago
  As already pointed in that discussion thread, there are standards for Unicode identifiers, e.g. RFC 8264 and RFC 8265. All Unicode characters have types, like letter, digit, punctuation, mathematical operator and so on. The standard for identifiers allows in identifiers only certain types of Unicode characters and it defines rules for normalization and comparison of identifiers. So rules for handling Unicode identifiers have already been defined. Whoever wants this functionality should just implement the standards. One may have opinions whether this is worthwhile or not for a certain application, but strawman arguments about cat pictures and other impossible dangers are no longer valid.
  - _yb2s 2y ago
    Apparently humor is no longer valid?
nineteen999 2y ago
I have an affectionate place in my heart for Debian, the community is passionate, they have wonderful ideals, hell I even helped found a charity which distributes it on used PC's discarded by large companies to disadvantaged people over 20 years ago which is still running today. It was my favourite distro for a long time after I moved on from Slackware in the late 90's, I used it at home, I used it in my job at a small ISP on everything from x86 to Sun Sparc to DEC Alpha hardware. We are lucky in the Linux community to have them. I could care less about deriatives like Ubuntu, seems to be one too far removed. But over the years the bikeshedding and some of the poor technical decisions started to wear on me. The debconf approach of asking a million questions on install bothered me. In my current job we use it on small industrial ARM PC's and it does a great job there at a large scale distributed over a wide variety of environments and geographical area, scorching heat, freezing cold and everything in between. But that's easy because it's a single system image which we deploy to hundreds of devices and it only requires minimal customisation to perform the required tasks. But our datacenter servers remain RHEL for the simple reason ... the deployment and broad customisation process per server is easy, LDAP integration is straight forward and the customer wants to pay for support from the vendor even though we never use it. Security updates and bugfixes are delivered quickly and the vendors commitment to stability is commendable. It's a no brainer. More and more companies started to move their workloads to RHEL once it came out and unfortunately it just didn't make sense to bother with distributions outside of RHEL/Fedora for my personal use anymore, some sort of work/life balance is needed and I don't want to spend my personal computing time remembering all the idiosyncracies between different Linux distributions anymore. I would argue that Debian is pretty idiosyncratic and opinionated if you have come from more traditional UNIX systems in the 90's, while RHEL/Fedora more closely model an "evolution" of those classic systems if you like. It will be interesting to see what happens to RHEL in the coming years as Redhat becomes more and more absorbed into the IBM environment.
- finnthehuman 2y ago
  That's the reality of deploying professionally though. I've got a soft spot for debian from using it for over 20 years too, but choosing open source often means picking the vendor that accommodates the use case. Many products have the enterprise upsell of good LDAP/AD integration but that's just a nice to have when you're really buy it for the ability to call someone when shit goes sideways. And when you don't need the support net, it's often gonna be ubuntu because that's what most people are comfortable with. Or yocto if you're shipping a custom OS. And containers are so ephemeral and purpose specific it means distro doesn't matter as much. I'm still rooting for them the most. They're community based, an important upstream, and stable has never done me dirty. It's still my go-to for "I don't want to think hard about, or worry about this system."
jcarrano 2y ago
I don't get it? What's the purpose of changing the default rule in shadow-utils. Not only is it completely unnecessary and introduces risks for shell injections, it also risks introducing incompatibilities between Debian and any other system. I feel that there are already too many other things to fix to be wasting time in creating new potential bugs.
thway15269037 2y ago
Before opening this can of worms, can we finally address that there is a hard, hardcoded limit of 255 bytes per file name (folder name) in Linux? Yeah, 255 bytes, that is, like 63 japanese characters or emojis or maybe less. And in kernel, too, so you physically cannot correct this issue by using another filesystem or something. Before anyone asks: yes, these folders do occur in real life, and I tired of pretending that they do not.
cratermoon 2y ago
My take: user names are not strings, though they may be represented as strings. As such, a type, e.g. Username, would provide a constrained and consistent range of allowed values, much as a type like float32 allows (within IEEE 754 rules). It's time for programmers to stop treating everything that can be represented by a string as anything representable by a string type.
quectophoton 2y ago
chown is getting "fun", I guess.
jmclnx 2y ago
Company I work at moved to an ID like [A-Z]Employee-number. Moot point for them :
okasaki 2y ago
Aren't pretty much all devices nowadays owned by a single person? What's the user case for non-system usernames at all? Why not just "user" and "root"?
ipython 2y ago
This sounds like a security nightmare just waiting to happen. Nothing like embedding gigantic libraries like libicu into security critical code bases so you can do things like Unicode normalization and comparison functions on usernames.
biglost 2y ago
I'm the type off person who don't use spaces in files, usernames, directories or folders, i'm too old to keep fighting scripts, i would vote for ascii and spaces forbidden in almost everywhere. But the right thing is use UTF-8 and hooe for the Best

249 points by jwilk 2y ago

Perhaps it's time to agree upon how to Unicode in identifiers? The normalization, unprintable characters, confusing characters with same glyphs, etc. It's obviously problematic when everyone is doing it on their own.

magicalhippo 2y ago
As long as I can enter my Zalgo[1] username, I'm fine with your suggestion. [1]: https://en.wikipedia.org/wiki/Zalgo_text https://en.wikipedia.org/wiki/Zalgo_text
m000 2y ago
Good luck bringing everyone together. There's still a ton of Microsoft software that relies on the presence of the BOM [1], despite practically everyone else not using it. And bidirectional rsync between practically everything else and a Mac still requires `--iconv=utf-8,utf-8-mac` to avoid problems because of homographs. [1] https://en.wikipedia.org/wiki/Byte_order_mark https://en.wikipedia.org/wiki/Byte_order_mark
bayindirh 2y ago
The first bar to clear is "The Turkish Test"[0], then we can talk about Unicode. It'll smooth the rest of the process a lot. You can't guess how many workarounds I implement to make sure that a stray application doesn't get "ı" or "İ" in their naive codepaths, and start burning mid-flight (e.g.: Kodi, Pagico, some old Java programs, oh my...). [0]: https://blog.codinghorror.com/whats-wrong-with-turkey/ https://blog.codinghorror.com/whats-wrong-with-turkey/
- beardyw 2y ago
  The date format part is ridiculous. Americans are almost unique in using mm/dd/yyyy, so an assumption of that would be plain wrong.
  - bayindirh 2y ago
    Localization libraries handle these parts well, since date is same with Europe (and generally stored as time-date objects rather than pure strings). None of the number shenanigans cause problems since these numbers are always stored as IEEE754 or other decimal formats. Money is no problem as well. However, when you go through an upper() or lower() or anything which plays with capitalization, and if that data is being fed to a hash algorithm or anything which mucks with strings, boy, oh boy... The easiest way is to sanitize these programmatic parts with forced locale of en_US or plain old "C". If the strings is not facing to the user and never localized, just force its locale. It's the only sane way.
    - oblio 2y ago
      > since date is same with Europe Do you mean MM-DD-YYYY? No, the vast majority of Europe does DD-MM-YYYY in some form or another.
      - bayindirh 2y ago
        No, I mean DD-MM-YYYY. We use the same format with the vast majority of Europe.
        oblio 2y ago
        I'm confused, are you talking about the US? The US for sure does not default to DD-MM-YYYY.
        bayindirh 2y ago
        I'm talking about the Turkish Language, its locale and its peculiarities since it has letters "ı" and "İ". So, I'm talking about Turkish date format. Turkey uses DD-MM-YYYY format, like the most of the Europe.
  - a3w 2y ago
    https://xkcd.com/1179/ https://xkcd.com/1179/ I heard the US and A are moving to the hissing cat date format shown here.
throw0101a 2y ago
> Perhaps it's time to agree upon how to Unicode in identifiers? And then update all data structures that refer to them (like last and w/who, also NFS), as well as file formats (like cpio, tar, and pax which encodes ownership).
- maccard 2y ago
  Yes. Those formats have had 20 years since Unicode was standardised, and things like my terminal still routinely break when given “unexpected” inputs. Practically every other application can handle it.
layer8 2y ago
Unicode has provided a specification for Unicode identifiers since 2005: https://www.unicode.org/reports/tr31/ https://www.unicode.org/reports/tr31/
- rini17 2y ago
  Great! Is there a library for their validation? ICU seems to have only spoof checker for confusables.
  - rurban 2y ago
    libu8ident
    - westurner 2y ago
      rurban/libu8ident : https://github.com/rurban/libu8ident https://github.com/rurban/libu8ident : > unicode security guidelines for identifiers
  - westurner 2y ago
    ICU: International Components for Unicode: https://en.wikipedia.org/wiki/International_Components_for_Unicode https://en.wikipedia.org/wiki/International_Components_for_U... unicode-org/icu: https://github.com/unicode-org/icu https://github.com/unicode-org/icu Microsoft/ICU: https://github.com/microsoft/icu https://github.com/microsoft/icu IDN: Internationalized domain name: https://en.wikipedia.org/wiki/Internationalized_domain_name https://en.wikipedia.org/wiki/Internationalized_domain_name Punycode: https://en.wikipedia.org/wiki/Punycode https://en.wikipedia.org/wiki/Punycode IDN homograph attack: https://en.wikipedia.org/wiki/IDN_homograph_attack https://en.wikipedia.org/wiki/IDN_homograph_attack CWE-1007: Insufficient Visual Distinction of Homoglyphs Presented to User: https://cwe.mitre.org/data/definitions/1007.html https://cwe.mitre.org/data/definitions/1007.html GNU libidn/libidn2: https://gitlab.com/libidn/libidn2 https://gitlab.com/libidn/libidn2 Comparison of regular expression engines > Language features > Part 2; Unicode support: https://en.wikipedia.org/wiki/Comparison_of_regular_expression_engines#Part_2 https://en.wikipedia.org/wiki/Comparison_of_regular_expressi...
secondcoming 2y ago
Would punycode be suitable?

tiahura 2y ago

When you think about all the time, money and effort that have been wasted on Unicode...

kalleboo 2y ago
Yeah we should have all just stuck to Shift-JIS
Joker_vD 2y ago
Всеки трябва да използва латиница, абсолютно съм съгласен. After all, it's objectively the most perfect set of characters for any reasonable human language.
- febusravenga 2y ago
  Random cross-language-script observation. In Bulgarian, латиница ("latin alphabet") transliterated to latin alphabet is just "latinitsa" or "latinica". In Polish "cyrillic" is "cyrylica" - basically reverse.
pjc50 2y ago
What's your preferred solution for representing the CJK languages?
- tiahura 2y ago
  Computing did pretty well in the prior 50 years.
  - pjc50 2y ago
    That's not an answer. Be specific. How do you want to represent the 97k CJK characters?
    - vman81 2y ago
      I really don't want to be snarky or sarcastic, so I'll just be plain. Many people are unwilling or unable to understand a problem that doesn't affect them directly. Like - "UTF is woke" kind of people. They are out there.

card_zero 2y ago

> naming things is one of the hard things to do in computer science I've been thinking about that a lot lately. Code is text, it's arranged linearly, code has to be readable, identifiers are thus short strings that try to express short essays about the purpose of the variable or whatever it is, and then ideally there's a longer version of the essay in a comment, but not too long because that would clutter up the code as well (because it's text, arranged linearly). And we have code folding to tidy them up, for what good it does, and ideally an even longer version of the essay in documentation except nobody writes that. What if it wasn't text, and wasn't linear, and we didn't have an expectation that code should be strings of stupid over-terse names and hieroglyphic symbols? So I was thinking vaguely about investigating graphic-based programming, but it's probably worse, IDK. It could automatically assign arbitrary icons* instead of identifiers, and you could write tooltip-like comments to describe them as and when you want to, and everything could be laid out nicely with diagrams and different pages instead of like a text file. I suppose this is all merely cosmetic? The thing with the instance on code being written as strings of text feels very primitive, is all. It causes this problem. * Which doesn't solve the problem, I admit, because now you have to remember what the icons mean, but maybe that's easier?

jstanley 2y ago
I don't think remembering the meaning of icons is easier, because in order to think about it you have to be able to pronounce it inside your head. And code isn't just linear, it can be spread across multiple files in a directory tree, functions can can each other, etc.
- c22 2y ago
  > in order to think about it you have to be able to pronounce it inside your head. I'm not sure this is universal.
  - vidarh 2y ago
    Indeed, some people do not even have an inner voice, the same way some of us don't "see" things in our minds eye. Neither prevents you from thinking about words or visual objects.

mmsc 2y ago

I wonder how this will affect ssh. OpenSSH recently restricted more characters for valid usernames: https://github.com/openssh/openssh-portable/commit/7ef3787c84b6b524501211b11a26c742f829af1a https://github.com/openssh/openssh-portable/commit/7ef3787c8...

cedws 2y ago
This is a great example of how one poor decision, or one piece of code that is too liberal cascades into an avalanche of shitty workarounds.
throw0101a 2y ago
It should be noted that shell metacharacters are also not allowed under POSIX: * https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_437 https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j k l m n o p q r s t u v w x y z 0 1 2 3 4 5 6 7 8 9 . _ - * https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282 https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... (Hyphen forbidden as first character.)
linuxftw 2y ago
I think it will be fine. Everyone will quickly learn the lesson "Use something other than ASCII letters and numbers at your own peril." Similar to people who put spaces in file names, it should be a fire-able offense.
- deleted 2y ago
  [deleted]
- lexicality 2y ago
  any software that can't handle spaces in filenames is broken
  - Muromec 2y ago
    All of the software is broken (including security wise) all the time anyway.

dfranke 2y ago

Allowing purely numeric usernames seems like a terrible idea to me, because it creates ambiguity between what's a username and what's a UID. It's common for tools like ls or ps to display a username when one is found and fall back to displaying a UID if it isn't, and similarly tools like chown will accept either a UID or a username and disambiguate based on whether it's numeric or not. Now suppose there's a numeric username that doesn't match its own UID, but does match some other user's UID. It doesn't take a lot of imagination to see how this would lead to vulnerabilities.

throw0101a 2y ago
Talk to POSIX: > A string that is used to identify a user; see also User Database. To be portable across systems conforming to POSIX.1-2017, the value is composed of characters from the portable filename character set. The <hyphen-minus> character should not be used as the first character of a portable user name. * https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_437 https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... The "portable filename character set" is defined as: A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j k l m n o p q r s t u v w x y z 0 1 2 3 4 5 6 7 8 9 . _ - * https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282 https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... So only a hyphen as the first character is forbidden. Given that you can't necessarilly control where usernames come from (e.g., LDAP lookups), properly speaking your system has to handle everything anyway, even if you don't allow local creation.
- dfranke 2y ago
  Yes, I'm aware, and POSIX has many such bugs that make command input or output unavoidably ambiguous if certain unexpected characters are present that they didn't think to prohibit. A lot of the revisions that went into POSIX 2024 were aimed at fixing some of these, such as standardizing find -print0 and xargs -0. The fact that this one got overlooked doesn't mean it's a good idea to make the situation worse and harder for future POSIX revisions to address.
- bluGill 2y ago

huhtenberg 2y ago

Sound like a solution in search of a problem. And a disruptive solution with unclear side effects at that.

johnisgood 2y ago

> If a keyboard input system provides the former sequence of bytes, but the username is stored in the login infrastructure using the latter sequence of [bytes], then a naive comparison will not find the user "émollier" in the system. Unicode defines in Annex 15 a few normalization forms as a way to work around this problem. But a correct use of these normalization forms still requires coordination and standardization among all programs accessing the data. ICU could work, but adds an extra dependency, there is also GNU's libunistring.

resource_waste 2y ago

This is important because Debain-family is used on many servers? Debian seems to just squander resources on things a few powerful people care about. All my servers have been Debian-based, so I can't be too hard on them, but whenever I see someone recommend a Debian-family distro as a Desktop OS, I feel like I need to call the police.

perlgeek 2y ago

Just imagine how many poorly-written shell scripts will break when we suddenly allow dollars, quotes, backticks and the likes in username. Heck, even allowing spaces sound like horror to me. On the display side, I'm sure most tools that display usernames won't make it easy to see if there are leading or trailing whitespace characters, double blanks, tabs etc in usernames. This sounds like support hell to me.

gmuslera 2y ago
The problem could be old scripts or systems that doesn’t handle UTF-8 (that doesn’t need to be the ones where the username was defined). I’m not sure if I.e. the Bobby tables trick could be done with characters with UTF8 representation seeing them in pure ascii.
deleted 2y ago
[deleted]
Starlevel004 2y ago
Breaking shell scripts sounds like a good idea to me. The faster they die the better the world gets.
- Rygian 2y ago
  That's going to be a very bumpy road, even if everyone were to agree that the destination is appealing.
  - bigstrat2003 2y ago
    Yeah for better or for worse compatibility is king. I despise shell scripts, they are an absolute nightmare to work with and full of footguns. But they are so commonplace that people are not going to tolerate YOLO breaking changes.
  - raverbashing 2y ago
    Yeah I think ESH While we have more modern shells the fact that bash (or even sh) is the "common denominator" 30 yrs on is both good and awful We need a PowerShell for Linux

nmstoker 2y ago

Unfortunate ambiguous uses of the word drop throughout the otherwise excellent article

TimK65 2y ago
There are three uses of the word "drop," all of which are correct. The latter-day meaning of "drop" is an abomination.
- toast0 2y ago
  I dropped X off at Y. Then X dropped off the face of the map, never to be seen again. Many words and phrases in English are self-antonyms.
fargle 2y ago
> The src:shadow package had dropped a Debian-specific patch, shoot, that's evil. had not noticed this. i read this as "removed", not "was released". now idk. this pseudo-definition of dropped as "released" is beyond stupid. yikes!

miohtama 2y ago

I remember useradd and adduser when learning Linux and oh boy what a confusion it was... Why not just one command

account42 2y ago

Always fun to see people poke the Unicode dragon only to be dumbstruck by its true size as it stands up in preparation of engulfing them with the fire of unintended consequences.

beardygo 2y ago
Indeed. As a speaker of several languages, including RTL language (they haven't even considered the problems with RTL marks etc), I say stay with ASCII for usernames, keep UTF for full names. If restricted ASCII a-z is good enough for passport names worldwide, it's good enough for usernames.
- macbr 2y ago
  I’m confused – my name as written on my passport definitely contains non ASCII characters?
  - extraduder_ire 2y ago
    What is it in the machine-readable section at the bottom? My passport takes the apostrophe out of my name down there.
    - belorn 2y ago
      What is the point of a machine-readable name when there is a machine-readable passport number which should be unique for each issuing country? In this age I would assume that places which uses machines to read passports also are connected to international databases where the unique number is checked for validation. My country also mandated passport with chips in them for the last couple of decades, so by now there are no longer any valid passports without such chip. If I had to guess, it seems the machine-readable section is just backward compatibility for machines built during the period where people started doing machine reading of passports but had yet to started to put chips into them. (as a fun side note, smart phones can read the chip on passports and this is then used by some digital identity providers to establish identity on account creation, in combination with the phone camera).

abigail95 2y ago

if you cannot handle UTF-8 anywhere anything approaching text could be, your program is malformed and should be deprecated and removed. if you wrote code that couldn't handle bob;>/hacked in a username, you would and should be laughed at. why are we using this ancient stuff?

knorker 2y ago
It's not just programs. And it's not just semantics of all-numeric username. It's also whether you want usernames that you cannot type, nor possibly even render. Definitely you can't spell it to someone else. Who owns that file? Oh, it's right-to-left non breaking space smiley snowman Chinese sign for water, I love that guy!
- abigail95 2y ago
  If people want to set up a Debian environment where people are mixing RTL and Hanzi I see no reason for that to be prohibited. Debian has opinions but I disagree that they should extend that far. If my employee Zalgo-fies everything. I don't file a bug report with Debian. I just fire them.
  - Muromec 2y ago
    >If my employee Zalgo-fies everything. I don't file a bug report with Debian. I just fire them. Which such clearly north American attitude you can as well use ASCII for everything.
- Izkata 2y ago
  > Who owns that file? Oh, it's right-to-left non breaking space smiley snowman Chinese sign for water, I love that guy! This reminds me, around 10 years ago on the chat app we used at work, we were able to change our nicknames and I made mine start with a combining character instead of a regular character. No one could ping me, it broke that part of the UI when they tried.
- adrian_b 2y ago
  This thread like also the parent thread is full of comments which are completely outdated, because there already exist standards for Unicode identifiers and obviously they forbid such cases. See e.g. RFC 8264. Only a restricted set of characters is permitted in identifiers, mostly letters and digits. This is enough to write any user name, without allowing "smiley snowman Chinese sign for water" or other such nonsense.

chikere232 2y ago

oh yes, let's break things to gain nothing of value

gspr 2y ago
Perhaps nothing of value _to you_. I'll hazard a guess that your preferred username can be expressed in a small subset of ASCII? And to hell with everyone else?
- knorker 2y ago
  I'll hazard a guess that your preferred username can't be written by 99.99999% of the world, and would always have to be copy-pasted?
  - Ylpertnodi 2y ago
    Yeah, us foreigners, up to our usual tricks again.
    - knorker 2y ago
      By any definition of the word, I'm a foreigner. So if you meant to imply that I'm an American, you've guessed wrong.
  - gspr 2y ago
    My preferred login name, and in fact also my legal name, are ASCII safe. I've never personally had to fight computer systems built with ridiculous assumptions (in the domain of names, at least). What I'm exhibiting in this thread is called empathy for people who don't happen to land in that category.
    - knorker 2y ago
      Did you appreciate my comment's real meaning, that one can have empathy for everyone else too, who need to work with this data? Another solution is, of course, to give everyone usernames they don't want. E.g. your user name is your employee number. Or your department plus three letters from your name that happen to make your username unique. I've seen both. I also extend my empathy to those who, after choosing a username nobody can type, don't get invited to meetings, don't get messages, are never referred to as the domain expert, because how can I add "习近平" to a meeting, if I can't even search for their username? Say even "mårten" is not showing up when I search my email, because those are not the same letters as "marten". Meh, emacron knows almost as much on this topic, so I'll just invite him instead. If you're already powerful enough, then people will be forced to copy-paste "mårten". If you're not already powerful enough, then you don't get invited at all, because that doesn't scale.

codedokode 2y ago

Don't you think that it would be better to get rid of usernames in UI? They only provide unique data for fingerprinting and do almost nothing useful on a single-user system. Wouldn't it be better to simply have a default name like "primary user" or "main user" for the first user and skip one step in installation process? Also it frees you from typing a username on login for a single-user system.

eviks 2y ago
Single user systems can just not ask for a username if there is only one, they control the UI

knorker 2y ago

So in the future I may not be able to even type the name of another user? Admins and other users not being able to type usernames sounds very bad. And I say that as someone whose native language has more letters than English.

zvr 2y ago

Most people are too young to remember that when you typed your username in all-caps in the login prompt (because the CapsLock key was on by accident, for example), the login(8) program assumed you were in a connection that could only do 7-bit (upper case, but no lower case characters) and immediately switched the tty settings and you were then presented with a "\PASSWORD: " prompt.

roelschroeven 2y ago
Don't you mean 6-bit? 7-bit ASCII supports lower case characters. Or maybe there are other 7-bit character sets that don't have lower case characters and it was one of those?
- jks 2y ago
  PETSCII? On the Commodore 64 you could press the Commodore key and Shift together to change character sets between lowercase and the graphical characters. But the Unix login thing might have been because of teletypes? https://www.columbia.edu/cu/computinghistory/teletype/ https://www.columbia.edu/cu/computinghistory/teletype/ claims that ASR 33 used 8-bit ASCII but was uppercase only - not sure if the "8-bit" claim can be true. On some Unix (and Linux) systems, you can still enter a kind of retro mode with "stty olcuc iuclc" (output lowercase to uppercase, input uppercase to lowercase) and turning on Caps Lock.
- zvr 2y ago
  You are of course correct that 7-bit ASCII includes lower case characters. I don't think there exists "6-bit ASCII", but the original ASCII did not have lower case (the slots were empty). We're talking early '60s here. I'm not even sure it was only about ASCII. I suppose I should have written a more generic "character set" (which supports or not lower case characters) rather than "7-bit". In the cases where you could only communicate in a single case (upper), you typed the commands in the usual letters (e.g., "LS") and capital letters were designated by a preceding backslash (e.g., "ECHO \JOHN \DOE"). That's why you were seeing the "\PASSWORD: " prompt, the initial letter was capital (as it still is). Just for fun, I checked my current Debian system. The getty(8) command still supports it: -U, --detect-case Turn on support for detecting an uppercase-only terminal. This setting will detect a login name containing only capitals as indicating an uppercase-only terminal and turn on some upper-to-lower case conversions. Note that this has no support for any Unicode characters.

soneil 2y ago

This reminds me of the systemd bug where usernames starting with a digit were mishandled (#15141). It seems to me like something that "should" be relaxed, but we need to have high confidence in the entire foodchain. adduser seems like the last place it should be changed, not the first - anyone requiring "enough rope" is already served by useradd.

hwc 2y ago

My work machine uses my complete email address as a user machine (this was a done by someone in the IT department). Vim gets confused when I use the `gf` command to open a path that contains an '@' character in it.

bjourne 2y ago

Honestly, it is super brain-dead that Linux and other operating systems still have such massive problems with "special" characters. Just the other day I had to help someone who had trouble building. The cause turned out to be that they had dropped filenames with parentheses in the source directory which, apparently, confused bash which make relies on. Such trash is everywhere on Linux systems. Eventually you learn to only use [a-zA-Z0-9-_.] in names because anything else will inevitably confuse some tool or another (even capital letters can be a PITA)... I so wish someone would take it upon themselves to clean up this mess, but it's probably too much work and too many who are nay-sayers conditioned to it who don't see the need for changes.

hiccuphippo 2y ago

As someone who needs non-ascii characters to write my name: please don't. You are making things worse just to be "courteous" about something we don't care about and will actually be annoyed at if we have to find how to write a letter in the keyboard or worse case scenario, figure out how to change the layout to the correct one before I even logged in.

jks 2y ago
Likewise. My last name contains a non-ascii character. In ~2009 I started at a company whose admin conveniently set up an account for me on their Ubuntu server... on which no-one could then log in locally because the login manager crashed when trying to display the list of users. I logged in via ssh and changed my name to the nearest ASCII equivalent. I always feel slightly worried on sites that demand that I give my full legal name (such as the US ESTA form), and then refuse to handle it because it includes "illegal" characters.
- ASalazarMX 2y ago
  This has happened to me with passwords containing foreign characters. The system would accept it, but further logons would be impossible. Now I always strip diacritics to be safe.
  - jks 2y ago
    A friend mentioned using control characters in passwords... like ^F and ^B, but not ^C because that's the interrupt character. Feels vaguely risky to me (does ^U empty the line? does ^W delete the last word? does your terminal emulator do some weird encoding like it does for cursor keys?) but if it works, why not?
    - jowea 2y ago
      I suspect I have run into a couple bugs because of password generators putting characters that some backend system cannot process in the password. Halfwish they just did DKWhhjwqjkwqjmHSJKHAIUHQwdmlsadkl instead.

SuperSandro2000 2y ago

They are clearly bored and want to start a year long bug hunt through half of unix

Muromec 2y ago
That sounds like a good kind of bored and bug hunting through half the unix sounds like fun too.
- db48x 2y ago
  Agreed! I’m half tempted to join the hunt myself just for kicks…

kej 2y ago

I wonder if it would work to do something like the punycode system for internationalized domain names. Shell scripts could handle a name like `xn--0civ130n` just fine, and user-facing utilities could choose to convert that to :sparkle::unicorn: when appropriate. The same homograph protections would probably work, as well.

dsr_ 2y ago

I will remind everyone that there are a minimum of three identifiers here. The UID, which is an integer. Ownership resides here; it's the primary key. Can be used by programs. The username, each of which must be unique and maps to one UID -- but multiple usernames can map to the same UID. Used by humans and programs to login. The GECOS field, or "human readable name", which is only used as a display label. Some systems include a structure inside this for additional info like phone number, office number, or similar". I don't think anyone would object to UTF-8 here.

seu 2y ago

The fact that this whole discussion happens in english, partially explains why there is a discussion at all. The whole problem could have been avoided if the development of computers had been a more international effort.

alterom 2y ago
[flagged]
alterom 2y ago
>The fact that this whole discussion happens in english, partially explains why there is a discussion at all. The whole problem could have been avoided if the development of computers had been a more international effort. Your comment makes for a solid example of why DEI efforts have a very practical side to it, and actually makes solid sense in the 21st century.

seiferteric 2y ago

OMG Can't believe this, I ran into this exact thing at my last job. We discovered a security vuln in several of our services because we were accepting unsanitized usernames, but since we and doing things with them (passing them to scripts etc.) but only after passing them to useradd/usermod etc so we thought they were safe, and of course you could put in things like ";" and "&", ">" etc and do whatever you want. I discovered that debian DISABLED the username sanity checks and could not believe it. anyway I installed a patched version as well as sanitized input and other stuff to resolve the issue.

IshKebab 2y ago

> Most Debian users don't work with useradd, or groupadd, directly. Instead, Debian has long supplied its own adduser (and addgroup) utilities, originally written by founder Ian Murdock. These act as simpler front ends to useradd One of the dumbest things Debian has done.

rurban 2y ago

They are so stupid, I cannot believe! Names are identifiers, and such need to stay identifiable. There exist unicode security guidelines and rules for identifiers, they don't know about. My libu8ident library would help with that.

_yb2s 2y ago

Clearly we should open up usernames to be an unlimited size set of mixed data types: e.g. the first “character” could be a hand drawn picture of a cat, the second the entire text of the US constitution in unicode, and so on. We could then extend this flexibility to filenames, passwords, and Unix commands. Internally, this could involve replacing all text strings with folders on a filesystem where you can put any files you want in any desired order. /s

adrian_b 2y ago
As already pointed in that discussion thread, there are standards for Unicode identifiers, e.g. RFC 8264 and RFC 8265. All Unicode characters have types, like letter, digit, punctuation, mathematical operator and so on. The standard for identifiers allows in identifiers only certain types of Unicode characters and it defines rules for normalization and comparison of identifiers. So rules for handling Unicode identifiers have already been defined. Whoever wants this functionality should just implement the standards. One may have opinions whether this is worthwhile or not for a certain application, but strawman arguments about cat pictures and other impossible dangers are no longer valid.
- _yb2s 2y ago
  Apparently humor is no longer valid?

nineteen999 2y ago

I have an affectionate place in my heart for Debian, the community is passionate, they have wonderful ideals, hell I even helped found a charity which distributes it on used PC's discarded by large companies to disadvantaged people over 20 years ago which is still running today. It was my favourite distro for a long time after I moved on from Slackware in the late 90's, I used it at home, I used it in my job at a small ISP on everything from x86 to Sun Sparc to DEC Alpha hardware. We are lucky in the Linux community to have them. I could care less about deriatives like Ubuntu, seems to be one too far removed. But over the years the bikeshedding and some of the poor technical decisions started to wear on me. The debconf approach of asking a million questions on install bothered me. In my current job we use it on small industrial ARM PC's and it does a great job there at a large scale distributed over a wide variety of environments and geographical area, scorching heat, freezing cold and everything in between. But that's easy because it's a single system image which we deploy to hundreds of devices and it only requires minimal customisation to perform the required tasks. But our datacenter servers remain RHEL for the simple reason ... the deployment and broad customisation process per server is easy, LDAP integration is straight forward and the customer wants to pay for support from the vendor even though we never use it. Security updates and bugfixes are delivered quickly and the vendors commitment to stability is commendable. It's a no brainer. More and more companies started to move their workloads to RHEL once it came out and unfortunately it just didn't make sense to bother with distributions outside of RHEL/Fedora for my personal use anymore, some sort of work/life balance is needed and I don't want to spend my personal computing time remembering all the idiosyncracies between different Linux distributions anymore. I would argue that Debian is pretty idiosyncratic and opinionated if you have come from more traditional UNIX systems in the 90's, while RHEL/Fedora more closely model an "evolution" of those classic systems if you like. It will be interesting to see what happens to RHEL in the coming years as Redhat becomes more and more absorbed into the IBM environment.

finnthehuman 2y ago
That's the reality of deploying professionally though. I've got a soft spot for debian from using it for over 20 years too, but choosing open source often means picking the vendor that accommodates the use case. Many products have the enterprise upsell of good LDAP/AD integration but that's just a nice to have when you're really buy it for the ability to call someone when shit goes sideways. And when you don't need the support net, it's often gonna be ubuntu because that's what most people are comfortable with. Or yocto if you're shipping a custom OS. And containers are so ephemeral and purpose specific it means distro doesn't matter as much. I'm still rooting for them the most. They're community based, an important upstream, and stable has never done me dirty. It's still my go-to for "I don't want to think hard about, or worry about this system."

jcarrano 2y ago

I don't get it? What's the purpose of changing the default rule in shadow-utils. Not only is it completely unnecessary and introduces risks for shell injections, it also risks introducing incompatibilities between Debian and any other system. I feel that there are already too many other things to fix to be wasting time in creating new potential bugs.

thway15269037 2y ago

Before opening this can of worms, can we finally address that there is a hard, hardcoded limit of 255 bytes per file name (folder name) in Linux? Yeah, 255 bytes, that is, like 63 japanese characters or emojis or maybe less. And in kernel, too, so you physically cannot correct this issue by using another filesystem or something. Before anyone asks: yes, these folders do occur in real life, and I tired of pretending that they do not.

cratermoon 2y ago

My take: user names are not strings, though they may be represented as strings. As such, a type, e.g. Username, would provide a constrained and consistent range of allowed values, much as a type like float32 allows (within IEEE 754 rules). It's time for programmers to stop treating everything that can be represented by a string as anything representable by a string type.

quectophoton 2y ago

chown is getting "fun", I guess.

jmclnx 2y ago

Company I work at moved to an ID like [A-Z]Employee-number. Moot point for them :

okasaki 2y ago

Aren't pretty much all devices nowadays owned by a single person? What's the user case for non-system usernames at all? Why not just "user" and "root"?

ipython 2y ago

This sounds like a security nightmare just waiting to happen. Nothing like embedding gigantic libraries like libicu into security critical code bases so you can do things like Unicode normalization and comparison functions on usernames.

biglost 2y ago

I'm the type off person who don't use spaces in files, usernames, directories or folders, i'm too old to keep fighting scripts, i would vote for ascii and spaces forbidden in almost everywhere. But the right thing is use UTF-8 and hooe for the Best