7 ms·

Here are some: * Doesn't scale. Having passwords in a plain text file is not a scalable solution for users directory. Can probably go up to a hundred users,

Here are some: * Doesn't scale. Having passwords in a plain text file is not a scalable solution for users directory. Can probably go up to a hundred users, but not much more. * In computer clusters you want user identity to "stick" to the user when they use multiple machines, containers etc. That's why you have LDAP... but it doesn't help all that much because user id is encoded into the file system (huge mistake...) which makes it very difficult to contain users to things they should control. If your only mechanism was the /etc/passwd, it would mean you'd have to constantly synchronize this file across all those machines and containers you have.

hiAndrewQuinn 3y ago
To anyone reading this and thinking "yeah dummy, of course it doesn't scale because you're not supposed to store passwords in plain text in the first place" I'll direct you to Chapter 7ish of The Linux Programming Interface. If you look in your /etc/passwd right now, you'll almost certainly see a single "x" where the (EDIT: no, it was still encrypted!) password originally was - nowadays that single "x" is an instruction to go look in /etc/shadow instead, for the salted hash of the password you're trying to check. I think this minimizes the number of users who need read permissions to /etc/shadow, and the amount of time they need it for. This has been your seemingly useless bit of Linux trivia for today. :)
- dsr_ 3y ago
  plaintext vs plain text unencrypted vs unstructured Of course, unstructured is also incorrect; the passwd and shadow files have structured records, one per line.
  - thwarted 3y ago
    …and being structured, the passwd file content should be accessed with the getpwent family of functions.
    - bhawks 3y ago
      Which unfortunately are not thread safe.
  - neuromanser 3y ago
    "unencrypted" is normally written as "cleartext". "plaintext" means "(readable / intended to be read) without a special viewer". Your.ssh /id_rsa is plaintext but not cleartext.
- vajrabum 3y ago
  /etc/shadow was born not because /etc/passwd had a plain text password but because the hashes became crackable and /etc/passwd is a public read file. Linux has never had them. Here's the man page indicating encrypted passwords for Unix v7 /etc/passwd release in 1979: https://man.cat-v.org/unix_7th/5/passwd https://man.cat-v.org/unix_7th/5/passwd
  - hiAndrewQuinn 3y ago
    Whoops! My bad, this is an even better bit of trivia. My mistaken memory really sells the underlying point that everything old is new again.
  - hinkley 3y ago
    I have a vague recollection of my 20 floppy of Slackware already having /etc/shadow. That would have been fall of 92 or winter 93, based on where I was living at the time.
    - mmcgaha 3y ago
      I have a vague recollection of being given the choice on a 90s vintage distribution with some warning about security and password length if I did not use shadow passwords. At some point in the early 2000s we started authenticating regular users against AD but the shadow file was still there for root.
      - hinkley 3y ago
        We had a couple of labs of sparcstations that just went away a couple of times a year because something bad would happen with all of the NFS mounted partitions and they'd have to turn the cluster on one box at a time to prevent thundering herd issues with NFS. I think they may have been mounting parts of /etc as well. People get the idea that managing accounts for a cluster of boxes should be centralized. It's all fun and games until the network mount disappears.
    - fuzztester 3y ago
      Shadow file was definitely from quite a while ago. Can't remember exact date, but might have been around time of SVR4 intro. I know because I remember going "ugh", but without investigating the reason why it (shadow) was introduced :) - which was of course wrong on my part.
- chasil 3y ago
  That was not a "plaintext password," it was a DES hash (from 7th edition onwards). This is the same format used by the classic htpasswd utility. https://en.wikipedia.org/wiki/Crypt_(C)#Traditional_DES-based_scheme https://en.wikipedia.org/wiki/Crypt_(C)#Traditional_DES-base...
dizhn 3y ago
This sounds like a completely unrelated thing and you are not constrained by the plain text password/shadow file for scale. NIS existed for many decades. You can even use Active Directory (or samba) for authentication and user management. But the article is not about this at all.
- crabbone 3y ago
  This shows you either didn't read what you replied to or don't understand the subject. This file is the public interface of the Linux system to everyone who wants to get information about users on the system. It doesn't matter that alternative tools exist: they were already mentioned in the post you replied to. It's not the point...
  - dizhn 3y ago
    If I didn't understand I'd be grateful if you could explain it. As far as I know that file does not get referenced for the users in an external directory server. That's how the systems scale without needing to put the users in the file. Aren't we talking about a high number of users (and their authorization levels) when talking about scalibility in this case?
  - Phrodo_00 3y ago
    > This file is the public interface of the Linux system to everyone who wants to get information about users on the system No it isn't. PAM is. The password file is only one of the places where users might be defined.
Attummm 3y ago
It may be old and not particularly appealing, but LDAP has been serving that role effectively at many companies.[0] Using a terminal remains standard practice for sysadmins and devops.[1] I believe there's some confusion in both the article and the comment between authentication and authorization. LDAP is fully equipped to handle both tasks. [0]https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system-level_authentication_guide/ldap_servers https://access.redhat.com/documentation/en-us/red_hat_enterp... [1]https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-co...
- crabbone 3y ago
  [flagged]
naasking 3y ago
> Doesn't scale. Having passwords in a plain text file is not a scalable solution for users directory. Can probably go up to a hundred users, but not much more. This simply cannot be true. A users directory that is frequently read will be in cache. The file is also fairly simple to parse. Even on old hardware you should be able to parse thousands of users.
- pseudostem 3y ago
  Exactly, even large textfile based DNS servers have capability to "compile" the textfile to a db file for faster access.
  - crabbone 3y ago
    So what if they do? This file is a public interface exposed by Linux to other programs. So what if Linux caches its contents when eg. Python interpreter on launch will read this file. And it's not coming from some "fake" filesystem like procfs or sysfs. It's an actual physical file most of the time.
    - fsckboy 3y ago
      the python interpreter reads /etc/passwd on launch? i guess it's looking for home directory
- wang_li 3y ago
  It's more of a problem of the number of nodes your users will be accessing than the number of users. It's a PITA to make mechanisms for password changes, for adding and removing users, for unlocking accounts, etc. when you have a distinct file on each server that needs to be changed. As well as adding new nodes to the environment and bringing them up to date on the list of users and etc.
  - naasking 3y ago
    Fair point, networking always makes things more complicated. I'm not sure it's really that much more complicated though. Like any concurrency problem with shared state, a single-writer and multiple readers keeps things simple, eg. a single host is authoritative and any changes are made there, and any file changes trigger automated scripts that distribute the changed file over NFS, FTP, etc. As long as new nodes start with the same base configuration/image this seems manageable. Simple to do with inotify, but even prior to that some simple C programs to broadcast that a change is available and listen for a change broadcast and pull the changed file is totally within the realm of a classic sysadmin's skillset.
hinkley 3y ago
My school had 30k students, including grad and doctoral students. When they gave students shell accounts, they tried to put them all onto a single Sequent box. They got my incoming class, the following, and anyone previous who asked for one onto that box. I’m pretty sure it had an /etc/password file, and would have had about 8-10k people on it. After that they gave up. Even with aggressive ulimits it was too hard, and each new class was apportioned to a separate Sparc. Which was a shame because we learned an awful lot about Unix administration from users pranking each other, figuring out what they did and how they did it, protecting yourself and retaliating (some people would prank others without first protecting themselves). Made it a lot harder chatting with people from other classes as well. For electives and humanities you weren’t always in classes with people your exact age. They could be ahead of you or behind.
pjc50 3y ago
> user id is encoded into the file system This is kind of unavoidable, but you do have 32 bits to play with. Windows did it slightly better with the SID: https://learn.microsoft.com/en-us/windows-server/identity/ad-ds/manage/understand-security-identifiers https://learn.microsoft.com/en-us/windows-server/identity/ad... > which makes it very difficult to contain users to things they should control It's not the file system that's the problem here, it's that "everything is a file" is not true for a whole bunch of important stuff that you might want to apply access control to on a UNIX system. Such as the right to initiate TCP connections. This sort of thing is why containers are so popular. NIS and LDAP do let you have a large number of users. Heck, we managed a few thousand users in /etc/password back when I was running https://www.srcf.net/ https://www.srcf.net/ .. in 2000.
- TFortunato 3y ago
  > it's not the file system that's the problem here, it's that "everything is a file" is not true for a whole bunch of important stuff that you might want to apply access control to on a UNIX system I wonder if there has ever been an attempt to really lean into, and push the limits of sticking with the "everything is a file" philosophy in this realm. I.e. how far could you get with having special files for fine grained permissions like "right to initiate a TCP connection", and making access control management be, essentially, managing which groups a user belonged to?
  - tenebrisalietum 3y ago
    I think Hurd and Plan 9 take the EIAF further.
  - generalizations 3y ago
    I think that was Plan 9.
  - jampekka 3y ago
    Plan 9 probably took this the furthest. Sad it didn't take off. https://en.m.wikipedia.org/wiki/Plan_9_from_Bell_Labs https://en.m.wikipedia.org/wiki/Plan_9_from_Bell_Labs
generalizations 3y ago
I've wondered why we don't have a passwd.d folder, the way we do with other things in the UNIX filesystem, with individual user accounts represented by individual files. Could even retain the same line-oriented format, just stored separately.
- sznio 3y ago
  I think a directory of 10000 files is worse than a file of 10000 lines.
  - generalizations 3y ago
    Maybe, but account synchronization across machines might be a lot simpler and easier.
marginalia_nu 3y ago
> * Doesn't scale. Having passwords in a plain text file is not a scalable solution for users directory. Can probably go up to a hundred users, but not much more. Why not? A file 100,000 line file will only take a moment to scan.
eternityforest 3y ago
That's the first understandable explanation I've ever heard of what exactly LDAP is and what it's for!
- dizhn 3y ago
  If you have services with even a few users you should set up LDAP (openldap is fine) and then experiment with it. LDAP is pretty old and very well supported overall. At one point I configured ldap for postfix virtual hosting. I chose LDAP rather than a database backed solution because of widespread first class support everywhere. The same directory later enabled me to use a ton of other things including SSO. You're always finding new ways to use it once you have it. It's a great skill to have and is nowhere near as complicated as people make it out to be, including live replication.

hiAndrewQuinn 3y ago

To anyone reading this and thinking "yeah dummy, of course it doesn't scale because you're not supposed to store passwords in plain text in the first place" I'll direct you to Chapter 7ish of The Linux Programming Interface. If you look in your /etc/passwd right now, you'll almost certainly see a single "x" where the (EDIT: no, it was still encrypted!) password originally was - nowadays that single "x" is an instruction to go look in /etc/shadow instead, for the salted hash of the password you're trying to check. I think this minimizes the number of users who need read permissions to /etc/shadow, and the amount of time they need it for. This has been your seemingly useless bit of Linux trivia for today. :)

dsr_ 3y ago
plaintext vs plain text unencrypted vs unstructured Of course, unstructured is also incorrect; the passwd and shadow files have structured records, one per line.
- thwarted 3y ago
  …and being structured, the passwd file content should be accessed with the getpwent family of functions.
  - bhawks 3y ago
    Which unfortunately are not thread safe.
- neuromanser 3y ago
  "unencrypted" is normally written as "cleartext". "plaintext" means "(readable / intended to be read) without a special viewer". Your.ssh /id_rsa is plaintext but not cleartext.
vajrabum 3y ago
/etc/shadow was born not because /etc/passwd had a plain text password but because the hashes became crackable and /etc/passwd is a public read file. Linux has never had them. Here's the man page indicating encrypted passwords for Unix v7 /etc/passwd release in 1979: https://man.cat-v.org/unix_7th/5/passwd https://man.cat-v.org/unix_7th/5/passwd
- hiAndrewQuinn 3y ago
  Whoops! My bad, this is an even better bit of trivia. My mistaken memory really sells the underlying point that everything old is new again.
- hinkley 3y ago
  I have a vague recollection of my 20 floppy of Slackware already having /etc/shadow. That would have been fall of 92 or winter 93, based on where I was living at the time.
  - mmcgaha 3y ago
    I have a vague recollection of being given the choice on a 90s vintage distribution with some warning about security and password length if I did not use shadow passwords. At some point in the early 2000s we started authenticating regular users against AD but the shadow file was still there for root.
    - hinkley 3y ago
      We had a couple of labs of sparcstations that just went away a couple of times a year because something bad would happen with all of the NFS mounted partitions and they'd have to turn the cluster on one box at a time to prevent thundering herd issues with NFS. I think they may have been mounting parts of /etc as well. People get the idea that managing accounts for a cluster of boxes should be centralized. It's all fun and games until the network mount disappears.
  - fuzztester 3y ago
    Shadow file was definitely from quite a while ago. Can't remember exact date, but might have been around time of SVR4 intro. I know because I remember going "ugh", but without investigating the reason why it (shadow) was introduced :) - which was of course wrong on my part.
chasil 3y ago
That was not a "plaintext password," it was a DES hash (from 7th edition onwards). This is the same format used by the classic htpasswd utility. https://en.wikipedia.org/wiki/Crypt_(C)#Traditional_DES-based_scheme https://en.wikipedia.org/wiki/Crypt_(C)#Traditional_DES-base...

dizhn 3y ago

This sounds like a completely unrelated thing and you are not constrained by the plain text password/shadow file for scale. NIS existed for many decades. You can even use Active Directory (or samba) for authentication and user management. But the article is not about this at all.

crabbone 3y ago
This shows you either didn't read what you replied to or don't understand the subject. This file is the public interface of the Linux system to everyone who wants to get information about users on the system. It doesn't matter that alternative tools exist: they were already mentioned in the post you replied to. It's not the point...
- dizhn 3y ago
  If I didn't understand I'd be grateful if you could explain it. As far as I know that file does not get referenced for the users in an external directory server. That's how the systems scale without needing to put the users in the file. Aren't we talking about a high number of users (and their authorization levels) when talking about scalibility in this case?
- Phrodo_00 3y ago
  > This file is the public interface of the Linux system to everyone who wants to get information about users on the system No it isn't. PAM is. The password file is only one of the places where users might be defined.

Attummm 3y ago

It may be old and not particularly appealing, but LDAP has been serving that role effectively at many companies.[0] Using a terminal remains standard practice for sysadmins and devops.[1] I believe there's some confusion in both the article and the comment between authentication and authorization. LDAP is fully equipped to handle both tasks. [0]https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system-level_authentication_guide/ldap_servers https://access.redhat.com/documentation/en-us/red_hat_enterp... [1]https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-co...

crabbone 3y ago
[flagged]

naasking 3y ago

> Doesn't scale. Having passwords in a plain text file is not a scalable solution for users directory. Can probably go up to a hundred users, but not much more. This simply cannot be true. A users directory that is frequently read will be in cache. The file is also fairly simple to parse. Even on old hardware you should be able to parse thousands of users.

pseudostem 3y ago
Exactly, even large textfile based DNS servers have capability to "compile" the textfile to a db file for faster access.
- crabbone 3y ago
  So what if they do? This file is a public interface exposed by Linux to other programs. So what if Linux caches its contents when eg. Python interpreter on launch will read this file. And it's not coming from some "fake" filesystem like procfs or sysfs. It's an actual physical file most of the time.
  - fsckboy 3y ago
    the python interpreter reads /etc/passwd on launch? i guess it's looking for home directory
wang_li 3y ago
It's more of a problem of the number of nodes your users will be accessing than the number of users. It's a PITA to make mechanisms for password changes, for adding and removing users, for unlocking accounts, etc. when you have a distinct file on each server that needs to be changed. As well as adding new nodes to the environment and bringing them up to date on the list of users and etc.
- naasking 3y ago
  Fair point, networking always makes things more complicated. I'm not sure it's really that much more complicated though. Like any concurrency problem with shared state, a single-writer and multiple readers keeps things simple, eg. a single host is authoritative and any changes are made there, and any file changes trigger automated scripts that distribute the changed file over NFS, FTP, etc. As long as new nodes start with the same base configuration/image this seems manageable. Simple to do with inotify, but even prior to that some simple C programs to broadcast that a change is available and listen for a change broadcast and pull the changed file is totally within the realm of a classic sysadmin's skillset.

hinkley 3y ago

My school had 30k students, including grad and doctoral students. When they gave students shell accounts, they tried to put them all onto a single Sequent box. They got my incoming class, the following, and anyone previous who asked for one onto that box. I’m pretty sure it had an /etc/password file, and would have had about 8-10k people on it. After that they gave up. Even with aggressive ulimits it was too hard, and each new class was apportioned to a separate Sparc. Which was a shame because we learned an awful lot about Unix administration from users pranking each other, figuring out what they did and how they did it, protecting yourself and retaliating (some people would prank others without first protecting themselves). Made it a lot harder chatting with people from other classes as well. For electives and humanities you weren’t always in classes with people your exact age. They could be ahead of you or behind.

pjc50 3y ago

> user id is encoded into the file system This is kind of unavoidable, but you do have 32 bits to play with. Windows did it slightly better with the SID: https://learn.microsoft.com/en-us/windows-server/identity/ad-ds/manage/understand-security-identifiers https://learn.microsoft.com/en-us/windows-server/identity/ad... > which makes it very difficult to contain users to things they should control It's not the file system that's the problem here, it's that "everything is a file" is not true for a whole bunch of important stuff that you might want to apply access control to on a UNIX system. Such as the right to initiate TCP connections. This sort of thing is why containers are so popular. NIS and LDAP do let you have a large number of users. Heck, we managed a few thousand users in /etc/password back when I was running https://www.srcf.net/ https://www.srcf.net/ .. in 2000.

TFortunato 3y ago
> it's not the file system that's the problem here, it's that "everything is a file" is not true for a whole bunch of important stuff that you might want to apply access control to on a UNIX system I wonder if there has ever been an attempt to really lean into, and push the limits of sticking with the "everything is a file" philosophy in this realm. I.e. how far could you get with having special files for fine grained permissions like "right to initiate a TCP connection", and making access control management be, essentially, managing which groups a user belonged to?
- tenebrisalietum 3y ago
  I think Hurd and Plan 9 take the EIAF further.
- generalizations 3y ago
  I think that was Plan 9.
- jampekka 3y ago
  Plan 9 probably took this the furthest. Sad it didn't take off. https://en.m.wikipedia.org/wiki/Plan_9_from_Bell_Labs https://en.m.wikipedia.org/wiki/Plan_9_from_Bell_Labs

generalizations 3y ago

I've wondered why we don't have a passwd.d folder, the way we do with other things in the UNIX filesystem, with individual user accounts represented by individual files. Could even retain the same line-oriented format, just stored separately.

sznio 3y ago
I think a directory of 10000 files is worse than a file of 10000 lines.
- generalizations 3y ago
  Maybe, but account synchronization across machines might be a lot simpler and easier.

marginalia_nu 3y ago

> * Doesn't scale. Having passwords in a plain text file is not a scalable solution for users directory. Can probably go up to a hundred users, but not much more. Why not? A file 100,000 line file will only take a moment to scan.

eternityforest 3y ago

That's the first understandable explanation I've ever heard of what exactly LDAP is and what it's for!

dizhn 3y ago
If you have services with even a few users you should set up LDAP (openldap is fine) and then experiment with it. LDAP is pretty old and very well supported overall. At one point I configured ldap for postfix virtual hosting. I chose LDAP rather than a database backed solution because of widespread first class support everywhere. The same directory later enabled me to use a ton of other things including SSO. You're always finding new ways to use it once you have it. It's a great skill to have and is nowhere near as complicated as people make it out to be, including live replication.