21 ms·

Debian still having trouble with merged /usr

232 points by bitcharmer 5y ago

rektide 5y ago
What a pity. Merged usr is so convenient to manage. I feel like most of the objections & problems relate to the transitory phase, to having to support both, to trying to install old packages. I was thinking Bookworm was committed to a merged usr, but sounds like the next release after, Trixie, will be the first pure usr-merged, when everyone should be on the new thing. For all the dpkg maintainer's griping, it sure doesnt seem like there are viable alternative implementations available to aliased directories.
mschuster91 5y ago
So that is the reason why that weird "unsupported" message showed up at a system I was upgrading some time ago. What I don't get... yes, it may be possible to get a dpkg database into an inconsistent state. But why is everyone hoping on a patch that may or may not appear to prevent these bugs, instead of deriving a way to fix a system with an inconsistent dpkg database?
- pabs3 5y ago
  The patch has already appeared, but it isn't yet in a mergeable state.
sph 5y ago
Democracy is pretty decent, but comes with some big flaws: tyranny of the minority, enormous amounts of back-and-forth to get anything done, big egos at any level can stop progress. Debian is the very example of it. Case in point: other distros forced the usr migration and very few problems were had. Debian put the idea through a committee, of course a minority wanted to keep the old behaviour so Debian decided to support both, guess what, supporting both means having two problems now.
- deleted 5y ago
  [deleted]
- LeoPanthera 5y ago
  > Debian put the idea through a committee, of course a minority wanted to keep the old behaviour That is specifically not what happened, and the article is quite clear about it. The Technical Committee voted unanimously for merged usr: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636#178 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636#178 The problem is that dpkg, an essential component of Debian, has specific issues with symlinks: https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_... So the issue is how merged usr is accomplished, not whether it is at all. Please don't spread this kind of misinformation.
  - sph 5y ago
    I'm quoting the article: "Debian took a more incremental approach, in part because it strives not to make wholesale changes to users' systems like those required by a flag-day upgrade to a merged /usr. In 2016, the ability to voluntarily switch to that scheme was added, then some attempts were made for newer versions of the distribution to be installed with a merged /usr by default. [...] The location of some files was being resolved at build time to point into /usr/bin (for example), but those files only existed in /bin on the non-merged systems." The fact that the committee took a decision that the core package manager can't support is baffling to me.
    - derefr 5y ago
      > The fact that the committee took a decision that the core package manager can't support is baffling to me. Why? Presumably the committee would be thereby deciding to drive development of the package manager toward whatever is required to support the high-level goal.
      - sph 5y ago
        Well, this hasn't happened, and now we have the situation mentioned in this article.
tzs 5y ago
From the original proposal from 2012 that is linked to in the article: > The primary commercial Unix implementation is nowadays Oracle Solaris. [...] By making the same change in Linux we minimize the difference towards the primary Unix implementation, thus easing portability from Solaris. I wonder what he means by "primary"? I'm reasonably sure than in 2012 OS X was used on more machines than Oracle Solaris.
- busterarm 5y ago
  They're talking about things that distro maintainers care about, namely servers. They are primarily not concerned with your workstation.
  - tzs 5y ago
    Was Solaris more used than HP-UX or AIX in 2012?
    - iso1210 5y ago
      I'm sure it was, but the numbers compared to Linux and even FreeBSD would be tiny.
    - busterarm 5y ago
      I'm still interviewing sysadmins from banks who have legacy systems to support and Solaris is probably 10:1 over HP-UX.
    - dagw 5y ago
      Can't speak about AIX, but it was massively more used than HP-UX. Solaris was still under pretty active development and adding new and cool features in 2012, while HP-UX seemed pretty much abandoned by HP. Itanium on HP-UX was HPs last big push on HP-UX, but after that failed to catch on they seemed to give up on the OS.
  - raphman 5y ago
    Lennart Poettering, the author of the proposal and creator of e.g., systemd, avahi, and pulseaudio, definitely cares about workstations, I'd say.
Cloudef 5y ago
Haven't used debian or debian based systems for many years. They do too much "automagic" things behind your back and thus tend to brick themselves, especially during major updates. Not really sure how they got so popular. The packaging ecosystem is also kinda mess.
- raphman 5y ago
  Huh? I have the opposite experience. Major upgrades always went flawlessly for me. With the rolling testing-Release on my workstation/laptop, I sometimes run into dependency problems during upgrades. But these can always be solved by a few standard techniques. Not really beginner-friendly, but quite reliable. I have had many more problems when upgrading Ubuntu installations.
  - pabs3 5y ago
    If you are running testing, it doesn't get security updates except through regular package migrations from unstable, so I recommend pulling in security updates from there manually or automatically. At least Firefox and Linux have regular security fixes in unstable, so you might want to use apt pinning to use the unstable versions. You can also use debsecan to automatically and temporarily pin security uploads to unstable using the technique mentioned here: https://wiki.debian.org/DebianTesting#Best_practices_for_Testing.2FSid_users https://wiki.debian.org/DebianTesting#Best_practices_for_Tes...
- rekoil 5y ago
  What killed Debian for me was when doing an upgrade of Apache and PHP it automatically restarted Apache temporarily without PHP configured, then the upgrade of PHP failed and left Apache running completely without PHP support for a while until someone noticed, and Apache happily kept serving .php files as text/plain during this time. A package manager should in my opinion leave daemon stopping/starting to the administrator, as they know their configuration better than any package manager ever can.
  - goodpoint 5y ago
Tor3 5y ago
I understand the problem dpkg is facing (can't know the canonical pathname for an object is, in some cases), but I have huge problems understanding why a symlink farm is preferable. The last thing I want to see on my Debian system is directories with hundreds or maybe thousands of symlink entries. There must be a way to let dpkg do its job if the directory-symlink approach is used instead, surely? (/bin -> /usr/bin)
- torstenvl 5y ago
  If we're linking hundreds of individual files on a single large partition, why not make them hard links and avoid the clutter?
  - rollcat 5y ago
    Dealing with hard links in the real world is a bit tricky. For example, "mv a b" is very different from "cat a > b". You have to do the equivalent of the latter in order to update a file in-place without updating every single hard link, which is problematic since writing to a file is not atomic the way an unlink+rename is. It's a bit of extra logic for the package manager to deal with, while a symlink is just a symlink.
  - s28l 5y ago
    It's entirely possible that /usr might be a separate partition from /, so hardlinks wouldn't work.
- codedokode 5y ago
  If you link /bin to /usr/bin then all binaries will be available under /bin, and developers will hardcode paths like /bin/python3. This will make programs non-portable to other distributiobns with separate /bin and /usr/bin. The link farm doesn't have this problem.
  - dTP90pN 5y ago
    Did that end up becoming a problem in all the other distros using aliased directories, for example Fedora which has 10 years of experience with this scheme? An quick look on Fedora 35 with $ grep -rP "(?!(?=(?'a'[\s\S]*))(?'b'\/usr(?=\k'a'\z)|(?<=(?=x^|(?&b))[\s\S])))\/bin\/(?!(sh|ksh|zsh|bsh|bash|dash|mount|umount|echo|true|false|ls|rm|mv|ln|cat))" /usr/bin doesn't really show any objectionable binaries being referred to via /bin (I count those I excluded in the negative lookahead as certainly non-objectionable). (I used the variable-length negative lookbehind [1] because I wanted to reduce false positives, but just scrolling through the results ended up being easier) [1] http://www.drregex.com/2019/02/variable-length-lookbehinds-actually.html http://www.drregex.com/2019/02/variable-length-lookbehinds-a... Alternatively, a simple grep --binary-files=text -rP '(?<!/usr)/bin/(?!sh|ksh|zsh|bsh|bash|dash|mount|umount|echo|true|false|ls|rm|mv|ln|cat)' /usr/bin to also include binary files does the job as well.
enriquto 5y ago
I'm surprised that they are pushing to remove /bin and keep /usr/bin. Why not the other way round? Let's keep our binaries out of /usr! (as a first step to remove /usr altogether, since it has a confusing name). I long for a day where PATH=/bin is all we need.
- _wldu 5y ago
  $HOME/bin is useful as well. I like that I can have my own software (that I wrote and compiled) in my home folder and don't need to ask a sys admin to install it for me.
  - CameronNemo 5y ago
    I use ~/.local/bin, but yeah using your homedir for as much as possible makes system administration easier for everyone.
- hashhar 5y ago
  Not everything is a binary. Also having everything system specific live under a single path means easy backups, easy immutable trees that can be A/B swapped for seamless and online updates with easy rollbacks. No more wondering where the various tools live since everything lives in /usr/bin and more.
  - zozbot234 5y ago
    The original point of a non-merged /bin (and /sbin) was to contain essential binaries for system recovery and early boot, that should be available even when a separate /usr/ partition is not yet mounted. Keep in mind that the approach of booting from initrd is quite non-idiomatic and used only for convenience wrt. a generic distro install; you're generally meant to recompile your kernel so that it can mount / directly at boot and find everything it needs there, without requiring to mount either a ramdisk or a separate partition. This means that early-boot stuff must live there in order to get the benefits of separate partitions in the first place.
    - 5y ago
isr 5y ago
Sigh ... gobolinux solved this quite elegantly over 20 years ago, and to this day most folks don't quite grok how simple and elegant their solution was.
- smcl 5y ago
  What was the solution, and why is is simple and elegant?
  - chalst 5y ago
    There's a rather old review on LWN: https://lwn.net/Articles/66290/ https://lwn.net/Articles/66290/ It's essentially like Macos's /Applications directory, except where Macos supports the traditional POSIX fhs via a hidden /private root directory, Gobolinux hides these directories with a filesystem kludge.
    - smcl 5y ago
      Nice, thanks! I'll give this a little read over lunch :)
      - 0des 5y ago
        I recently did something similar refactoring XDG endpoints to get rid of the standard home layout. I was quite surprised everything just worked, and there wasn't an issue of directories being recreated in their original locations or file templates being missing. Quite liberating if you ask me.
- Saint_Genet 5y ago
  Having potentially hundreds of copies of popular shared libraries on disk isn't particularly elegant when you find a bug in those libraries
  - rnhmjoj 5y ago
    I'm sure how you're getting to this conclusion. GoboLinux filesystem hierarchy just separates executables and shared objects by name and version [1]; unlike the standard FHS where everything goes into /usr/lib, /usr/bin, etc. For backward compatibility there are symlinks (that takes no disk space) to the standard directories, but these are hidden by the kernel so they don't show up and clutter your file manager. [1]: https://gobolinux.org/at_a_glance.html https://gobolinux.org/at_a_glance.html [2]: https://gobolinux.org/doc/articles/gobohide.html https://gobolinux.org/doc/articles/gobohide.html
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
iso1210 5y ago
> The /usr merge idea was first raised in "The Case for the /usr Merge" by Lennart Poettering in 2012. It came out of the systemd community Well that's a recipe for a long drawnout battle
- ungamedplayer 5y ago
  It is just people with emotional damage.
- jillesvangurp 5y ago
  As somebody who really does not care either way, all this just sounds more like moving problems rather than solving a problems. At least, I don't think there are many Linux systems where you can safely remove /bin from the path just yet. I just checked. My Manjaro install has a /bin and it is full of files that look like I would not want to lose them. So, they fixed the /bin to /usr/bin thing for some packages but clearly not all packages in Arch. I bet/blindly assume that's true in Fedora as well. I've always struggled with figuring out where stuff lives on the file system in different linux and unix derivatives. It's never where you expect/want it to be. I've used solaris, hp ux, mac os, and linux over the years. They all have similarly named directories, mostly for weird historic reasons. Is it /var/usr/god/knows/what or in /opt/foo/bar or in /usr/var/lib or /usr/share/lib. What about /etc, /usr/etc, /usr/local/etc? You can take almost any 1,2,3, and 4 symbol permutations of share, lib, var, bin, opt, and sbin and probably find something that expects files to exist under that path. Reducing the number of permutations would probably be helpful. People seem to just roll the dice and create some place where shit lives based on mostly just vague intentions, rules, heuristics, and interpretations of those associated with long dead unix variants from the nineteen eighties, weird naming conventions, and what not. So, what's the rule here? Some 'special' packages should install to /bin and some other not 'special' packages should install to /usr/bin? Why? Is there any agreement about what constitutes a 'special' package? Is there a good functional reason for having both directories? And then the whole bin/sbin distinction is kind of arbitrary as well. Some binaries are statically linked, others are not and require libraries to exist in yet more shared directories on a library path. It all boils down to users requiring a PATH variables (and, inevitably, a lot of other/similarly named variables). And of course the order of paths is also super relevant and kind of the point. So you look in /usr/bin before or after you look in /sbin? It's all a house of cards. Brittle by design and convention. The notion of taking a package and then fragmenting it over a multitude of shared directories is the problem that needs fixing. The role of a package maintainer is bridging those different notions of where stuff should live between different distributions with some convoluted scripts. It's a job that should not need doing and code that should not need to be written. The main differences between linux distributions boil down how none of them actually having solved this problem that ended up doing only loosely similar but clearly different things for mostly obscure reasons. They don't agree on where stuff should live, how it should be moved/copied/linked there, where and how things are configured, etc. Most of these differences are kind of arbitrary and petty. /sbin, /usr/bin, /usr/sbin, /bin, /usr/local/bin, etc. who cares? I pretty much need all of these in my path for things to work as intended. I don't see a good reason for more than 1 of these to exist. Apple kind of got this half right with the notion of mounting a package rather than installing it. Most applications install/uninstall via drag & drop. I always thought that was a neat idea. Of course, they then made it complicated by having /User/<uid>/Library/* and /Library/* directories anyway. So, most applications leave a lot of clutter there after you drag them to the trash-can. And they also have the usual contingent of unixy directories. And package managers like brew, macports, fink, and whatnot that sort of carved out different places in the filesystem where their stuff lives. So, I wouldn't go as far as saying that Apple solved the problem. But it does look like progress to me. Mounting stuff rather than fragmenting it all over the place is progress. Docker does this. And so do Flatpak and Snap. Flatpak and Snap are kind of tedious in their own way (e.g. opencl support is a PITA with Darktable and other packages that need that). But at least I have some stuff that I installed with those that actually works without having to be customized for every linux distribution.
richardfey 5y ago
Instead of all this drama the person that cannot see any way forward, despite a valid example of Ubuntu making this work, should be removed from the critical path by the instrument of a fork. Otherwise it will just be a matter of time before another bike-shedding, multiple years-spanning incident happens again. By creating a fork and proving that it works, the discussion becomes about whether to merge patches or adopt the fork rather than sterile "what if"s. Edit: added last paragraph
Zardoz84 5y ago
Giving that on Ubuntu this was done (and in other distros), and there isn't any issue ( I just noticed that I have a merged /usr for a long time), the dpkg maintainer it's being a jerk.
- josefx 5y ago
  As far as I can find there are issues, namely that installing a package with dpkg can leave the system in a bad state. If I understand it correctly dpkg normally prevents by checking the paths it modifies, which ends up non trivial if the paths are aliased. > the dpkg maintainer it's being a jerk. He is required to accept working patches, priority by the committee seems to be the removal of warnings about the broken support instead of actually fixing it.
  - Beltalowda 5y ago
    > He is required to accept working patches The thing is that no one wants to interact with a maintainer hostile to the very idea of your patch. Sure, they may be "required" to accept it, but it's not going to be a fun process for anyone involved. Part of Boccassi's concern with their patch is that it might not be merged at all due to what they perceive as "moving goalposts" and "excuses". I think that's not entirely an unreasonable concern, and no one likes to work on patches that will never get merged. I don't know what the path forward for Debian would be here. Things seem ... difficult. Getting some of the social tension solved and having people "kiss and make up" would probably be a good start, but that can only work if people are willing.
    - josefx 5y ago
      > The thing is that no one wants to interact with a maintainer hostile to the very idea His hostility seems to be based on actual reasons if it is true that other systems that performed the unification basically throw out any guarantees made by dpkg. Providing a patch that fixes that would get rid of those reasons. > Part of Boccassi's concern with their patch is that it might not be merged at all due to what they perceive as "moving goalposts" and "excuses". Bocassi is also the guy who claimed the first incomplete patch was rejected without further comment, which going by the article wasn't the case. So his opinion of the maintainer is at best misinformed, at worst intentionally deceptive. Also going by the article a merge can be forced by the committee. So this seems to be a non issue.
nickcw 5y ago
Just in case anyone is wondering why we had (or used to have) /bin and /sbin directories as well as /usr/bin /usr/sbin here is my understanding of it. It is because on old Unix systems there frequently wasn't enough space to store /usr on the root disk. Therefore /usr was a separate disk which might not be available during boot. This meant that you needed to put all the binaries you needed for boot in /bin and /sbin because /usr/bin and /usr/sbin might not be mounted yet. This no longer makes sense in the days of large disks which fit /usr just fine and initrds which can mount all the disks before kicking off the things which might need those binaries. In fact I think the initrd has taken the place of /bin and /sbin on a modern system containing copies of all the binaries needed to mount the disks (like mount and fsck).
- ntauthority 5y ago
  Adding to this historical reasoning, it is to be noted that in traditional Unix, /usr was pretty much the equivalent of /home in Linux systems: the parent directory for user home directories, i.e. 'usr'. The existence of /usr/bin can probably be explained by the reasoning that, just like nowadays Linux systems often have a really big /home partition, early Unix systems eventually ran out of space for whatever volume would hold /bin, and with 'reinstalling' not remotely as easy an option as it would be these days, storing an extra 'bin' etc. directory in the 'larger partition for user directories' became commonplace.
  - ink_13 5y ago
    Careful, you'll attract computing historians who will want to tell you that "usr" actually stands for "UNIX system resources"
    - Cockbrand 5y ago
      That's how I've learned it around 1995, and that the division between /[s]bin and /usr/[s]bin has the reason that the former are required for system boot, while the latter are mounted via NFS from a central server - why waste local disk space when these would be identical for every system anyway (in a homogenous environment, at least). While that made a lot of sense to me back then, I've become aware that it might not be absolutely accurate, though.
chx 5y ago
- deleted 5y ago
  [deleted]
- dbrgn 5y ago
  If you'd actually read the article, you'd see that the problem isn't merged /usr. Other distros moved to merged /usr just fine. Also, the decision to move Debian to merged /usr wasn't done by Poettering, it was done by the Debian technical committee. So please stop spreading such ad-hominem nonsense.
- 0des 5y ago
egberts1 5y ago
We can now kiss-gone the security for embedded world and its IoT-related security brethren. It is often the directory separation of /bin and /usr/bin that is used to denote the extent of firmware’s scope between these binaries that are require for a boot up and what are they require to support their applications. (Thanks, PDP-11). Such wonderful boundaries of IoT upgrade scope can and is often denoted as two-stage upgrade with /bin as read-only; now, not so much. This is yet another case of IoT system engineering design getting steamrolled by the diminutive desktop-mindsets despite progress by its systemd author. Try yanking that Ethernet cable from its physical socket and watch if your long-running deeply-stateful process can survive. (It won’t, unless you retrain it against KISS principle to ignore such netdev disappearance.)
- yjftsjthsd-h 5y ago
  > We can now kiss-gone the security for embedded world and its IoT-related security brethren. Is anyone using Debian for that kind of thing? When I hear embedded, I expect buildroot or at least Alpine Linux.
- WhyNotHugo 5y ago
  Using `/usr/bin/` readonly is also a thing on some few distros (e.g.: Fedora Silverblue), so maybe you want to consider that for your usecase. For device-local binaries (e.g.: not part of firmware), /usr/local/bin sounds like the right choice (also, somewhat in line what some BSDs do).
  - egberts1 5y ago
    This is just another direction to desktop-hive mentality with a focus toward minimizing security. Perhaps, it might be ideal and suitable to Windows-ize for your case; the security modeling of many designs, not so much.
    - WhyNotHugo 5y ago
      How does this minimise security? I don't see how Window is relevant; it doesn't have /usr nor /usr/bin.
upofadown 5y ago
>Improved compatibility with other Unixes/Linuxes... A quick check reveals that OpenBSD presently has separate /bin and /sbin. FreeBSD also has a separate /lib. Exactly what were the other Unixes/Linuxes that this was supposed to improve compatibility with? I hope was not just Solaris...
- faho 5y ago
  The idea is, if something hardcodes e.g. /bin/grep because it was written on a system where that's where grep is, it will suddenly start working because now grep is also accessible via /bin/grep. So it's compatibility in the sense of making things work (instead of failing over pointless differences), not in the sense of being the same.
- Beltalowda 5y ago
  OpenBSD also has /usr/X11R6, where it puts the X stuff (their X distro is named Xenocara). This is another piece of legacy from back when you might want to have both /usr/X11R2 and /usr/X11R3. These days the X11R6 directory actually contains X11R7 shrug. FreeBSD used to do this too, but got rid of it in FreeBSD 6 (I think? Maybe 7? About ten years ago). Both systems install packages to /usr/local.
  - tremon 5y ago
    Both systems install packages to /usr/local That's because "packages" in the BSD world (at least openbsd) are considered third-party options, the base system is considered static. Linux distributions don't really have the notion of a static base system with "package" addons, the distribution packages are the system.
solcloud 5y ago
You can't teach an old dog new tricks. Debian docker image has open issue for this for years too.
BlinkenBlinken 5y ago
Current state of the UNIX file hierarchy is a mess. Many incarnations of GNU/Linux systems didn't contribute to a more elegant yet simplified directory structure. Just pick the one you like the most and serves the purpose of your desires. As for merged /usr, recently using Arch Linux for desktop use. Works decent enough to getting things done.
- tenebrisalietum 5y ago
  Elaborate. I don't find the UNIX file hierarchy complicated, but there are some historical quirks and artifacts (like `/var/lib`). The difference between `/usr/X` and `/X` is about the worst of it IMHO. Everything important is usually at most 3 levels deep and the short names are very convenient (versus "C:\Users\xxx\Documents and Settings"). Compare to things like Windows: c:\windows\system32 that has 64 bit stuff in it, the whole c:\windows\syswow64 b.s., c:\Program Files (x86) and c:\Program Files, the hierarchy in the Windows registry which has "Windows", "Windows NT", "Microsoft Windows" (probably), etc.
  - jeroenhd 5y ago
    All file system hierarchies contain complicated rules and systems that have been built in by the people who designed them decades ago. I still don't know what I'm supposed to do with /usr/local, what the difference is between /usr/share and /usr/local/share, what the point of /opt is if programs install their files and dependencies in /usr(/local?)/lib anyway and why I have /usr/lib, /usr/lib32 and /usr/lib64 when only two directories and the right environment variables should suffice. There are subfolders in /dev that feel like they don't need to be subfolders. There's /tmp and three other places that contain temporary files that should get cleaned on boot. PID files appear strwen across /var/run and /tmp/<magic directory name>. /var can contain just about everything. Most of /var feels like it should actually be inside /var/spool but you're not going to see much in there except for a mail queue to nowhere on desktop Linux machines. Then we come to the XDG standard everybody just blatantly ignores that tries to bring order to the chaos that is program-generated files in the home directory. And then there's also snap. Snap looks at any directory convention, laughs, spits in your face for good measure, and creates a folder called "snap" wherever the fuck it wants to. I'd purge it from my system if the snap people hadn't convinced some tools I use to support it as the main distribution method. I'm sure there are guides out there that explain every directory and their purpose. I've read one of those guides, noticed that at least a third of the common programs I use clearly haven't read it, and forgotten the details already. The file hierarchy of a fully-fledged desktop Linux is kind of a mess, and that's just what you get when your core system is formed by combining the work of hundreds or thousands of volunteer projects.
BlinkenBlinken 5y ago
Current state of the UNIX file hierarchy is a mess. Many incarnations of GNU/Linux systems didn't contribute to a more elegant yet simplified directory structure. Just pick the one you like the most and serves the purpose of your desires. As for merged /usr, recently using Arch Linux for desktop use. Which accomplishes this by symlinking, which is ugly. Arch certainly is not perfect. Not sure if this is the way.
Decabytes 5y ago
I wish most *Unixes would go the Gobo Linux route with a Program, Users, System, Data, Mount file hierarchy that is just symlinks under the hood to the old hierarchy
- zokula 5y ago
- CameronNemo 5y ago
  Why rename? How does that benefit us? The only real difference gobo offers is separating program and system files, but I'm not even sure what the difference is there. Other than that: * /Users == /home * /Data == /var * /Mount == /mnt (?) so what do we get by breaking the world?
  - Decabytes 5y ago
    > so what do we get by breaking the world? In the the gobo documentation^1 section called "But what about Unix compatibility?" > The GoboLinux system layout seems to be a major departure from the Unix tradition. Does this mean all programs need to adjusted so that they work with the new layout? Fortunately, the answer is no. Through a mapping of traditional paths into their GoboLinux counterparts, we transparently retain compatibility with the Unix legacy. ~] ls -l /dev/null | cut -b 45- /dev/null ~] ls -l /bin/sh | cut -b 45- sh -> /Programs/Bash/4.4/bin/bash ~] ls -l /usr/include/stdio.h | cut -b 45- stdio.h -> /Programs/Glibc/2.24/include/stdio.h > There is no rocket science to this: /bin is a link to /System/Index/bin. And as a matter of fact, so is /usr/bin. And /usr/sbin... all "binaries" directories map to the same place. Amusingly, this makes us even more compatible than some more standard-looking distributions. In GoboLinux, all standard paths work for all files, while other distros may struggle with incompatibilites such as scripts breaking when they refer to /usr/bin/foo when the file is actually in /usr/local/bin/foo. > You may have noticed that the Unix paths did not show up in the system root listing in the very first example. They are actually there, but they are concealed from view using the GoboHide kernel extension. This is for aesthetic purposes only and purely optional, though: GoboLinux does not require modifications in the kernel or any other system components. But our users seem to like it a lot. :-) So it doesn't break the world. The faq^2 also indicates this wasn't a change to make Linux more newbie friendly, but in my own words I think it does. 1. https://gobolinux.org/at_a_glance.html https://gobolinux.org/at_a_glance.html 2. https://gobolinux.org/faq.html https://gobolinux.org/faq.html
zh3 5y ago
I've always liked the /usr distinction, with systems set up so they will at least boot without /usr mounted (at an absolutely minimum, a statically linked /bin/sh is enough to get a command prompt). This is particularly valuable for setups where a large number of PCs can mount /usr read-only from a remote server (we run quite a number of diskless x86 amd ARM systems like this).
- rektide 5y ago
  Bind mount your small system's /usr to /lower, mount your bigger /upper, and use overlayfs to combine them. Looking forward ro some blog posts discussing how to support this use case; it's a good one. I think usr-merge is so much more managable though that it's totally worth doing.
  - zh3 5y ago
    Must admit I'm surprised there don't seem to be more users out there doing exactly this. Makes installing a new program for everyone instant (put it in /usr, now available to all) while allowing personal customisations with the ability to reset the system to a pristine state in a second. Esp. combined with diskless, the computer hardware becomes irrelevant/disposable - just grab a fresh one, boot the same image and carry on (or boot your personal image at a hot desk).
- CameronNemo 5y ago
  Can you not use an initramfs to support mounting an ro NFS root? You could even ship the initramfs and kernel using tftp. Then the diskless systems just need enough storage for a bootloader like uboot or tianocore.
mmh0000 5y ago
The Filesystem Hierarchy Standard[0][1] is used by most Linux distributions as the authoritative source and description of the various system directories. As other commentators have already noted, the hierarchy on UNIX system has always been a bit of a mess, the FHS is an attempt to clean up while maintaining backwards compatibility. [0] https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html [1] https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard#History https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard#...
xyzzy_plugh 5y ago
As a Debian user in a previous life, and a current NixOS user, this whole debacle amuses me greatly, as I no longer give a shit about paths like /usr/bin or /bin, beyond a handful of binaries needed to bootstrap my environment. I've even moved those paths into a read-only filesystem on some systems!
- jeltz 5y ago
  This could happen in any distribution which does not have a powerful dictator at the top. Almost nothing about this issue has to do with paths. It is one maintainer of a core package who obstructs the Debian project which is enabled by Debian's distributed nature with no powerful dictator at the top.
- Ericson2314 5y ago
  Yeah came here to say the same. For them, silly tradeoffs between uniformity and what might be considered ambient authority or separation of concerns. For us? Nahhh best of both worlds.
smm11 5y ago
I thought Apple sort of did this already? (I know, OS X is not Linux.)
bumblebritches5 5y ago
deleted 5y ago
[deleted]

232 points by bitcharmer 5y ago

What a pity. Merged usr is so convenient to manage. I feel like most of the objections & problems relate to the transitory phase, to having to support both, to trying to install old packages. I was thinking Bookworm was committed to a merged usr, but sounds like the next release after, Trixie, will be the first pure usr-merged, when everyone should be on the new thing. For all the dpkg maintainer's griping, it sure doesnt seem like there are viable alternative implementations available to aliased directories.

mschuster91 5y ago

So that is the reason why that weird "unsupported" message showed up at a system I was upgrading some time ago. What I don't get... yes, it may be possible to get a dpkg database into an inconsistent state. But why is everyone hoping on a patch that may or may not appear to prevent these bugs, instead of deriving a way to fix a system with an inconsistent dpkg database?

pabs3 5y ago
The patch has already appeared, but it isn't yet in a mergeable state.

sph 5y ago

Democracy is pretty decent, but comes with some big flaws: tyranny of the minority, enormous amounts of back-and-forth to get anything done, big egos at any level can stop progress. Debian is the very example of it. Case in point: other distros forced the usr migration and very few problems were had. Debian put the idea through a committee, of course a minority wanted to keep the old behaviour so Debian decided to support both, guess what, supporting both means having two problems now.

deleted 5y ago
[deleted]
LeoPanthera 5y ago
> Debian put the idea through a committee, of course a minority wanted to keep the old behaviour That is specifically not what happened, and the article is quite clear about it. The Technical Committee voted unanimously for merged usr: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636#178 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636#178 The problem is that dpkg, an essential component of Debian, has specific issues with symlinks: https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_... So the issue is how merged usr is accomplished, not whether it is at all. Please don't spread this kind of misinformation.
- sph 5y ago
  I'm quoting the article: "Debian took a more incremental approach, in part because it strives not to make wholesale changes to users' systems like those required by a flag-day upgrade to a merged /usr. In 2016, the ability to voluntarily switch to that scheme was added, then some attempts were made for newer versions of the distribution to be installed with a merged /usr by default. [...] The location of some files was being resolved at build time to point into /usr/bin (for example), but those files only existed in /bin on the non-merged systems." The fact that the committee took a decision that the core package manager can't support is baffling to me.
  - derefr 5y ago
    > The fact that the committee took a decision that the core package manager can't support is baffling to me. Why? Presumably the committee would be thereby deciding to drive development of the package manager toward whatever is required to support the high-level goal.
    - sph 5y ago
      Well, this hasn't happened, and now we have the situation mentioned in this article.

tzs 5y ago

From the original proposal from 2012 that is linked to in the article: > The primary commercial Unix implementation is nowadays Oracle Solaris. [...] By making the same change in Linux we minimize the difference towards the primary Unix implementation, thus easing portability from Solaris. I wonder what he means by "primary"? I'm reasonably sure than in 2012 OS X was used on more machines than Oracle Solaris.

busterarm 5y ago
They're talking about things that distro maintainers care about, namely servers. They are primarily not concerned with your workstation.
- tzs 5y ago
  Was Solaris more used than HP-UX or AIX in 2012?
  - iso1210 5y ago
    I'm sure it was, but the numbers compared to Linux and even FreeBSD would be tiny.
  - busterarm 5y ago
    I'm still interviewing sysadmins from banks who have legacy systems to support and Solaris is probably 10:1 over HP-UX.
  - dagw 5y ago
    Can't speak about AIX, but it was massively more used than HP-UX. Solaris was still under pretty active development and adding new and cool features in 2012, while HP-UX seemed pretty much abandoned by HP. Itanium on HP-UX was HPs last big push on HP-UX, but after that failed to catch on they seemed to give up on the OS.
- raphman 5y ago
  Lennart Poettering, the author of the proposal and creator of e.g., systemd, avahi, and pulseaudio, definitely cares about workstations, I'd say.

Cloudef 5y ago

Haven't used debian or debian based systems for many years. They do too much "automagic" things behind your back and thus tend to brick themselves, especially during major updates. Not really sure how they got so popular. The packaging ecosystem is also kinda mess.

raphman 5y ago
Huh? I have the opposite experience. Major upgrades always went flawlessly for me. With the rolling testing-Release on my workstation/laptop, I sometimes run into dependency problems during upgrades. But these can always be solved by a few standard techniques. Not really beginner-friendly, but quite reliable. I have had many more problems when upgrading Ubuntu installations.
- pabs3 5y ago
  If you are running testing, it doesn't get security updates except through regular package migrations from unstable, so I recommend pulling in security updates from there manually or automatically. At least Firefox and Linux have regular security fixes in unstable, so you might want to use apt pinning to use the unstable versions. You can also use debsecan to automatically and temporarily pin security uploads to unstable using the technique mentioned here: https://wiki.debian.org/DebianTesting#Best_practices_for_Testing.2FSid_users https://wiki.debian.org/DebianTesting#Best_practices_for_Tes...
rekoil 5y ago
What killed Debian for me was when doing an upgrade of Apache and PHP it automatically restarted Apache temporarily without PHP configured, then the upgrade of PHP failed and left Apache running completely without PHP support for a while until someone noticed, and Apache happily kept serving .php files as text/plain during this time. A package manager should in my opinion leave daemon stopping/starting to the administrator, as they know their configuration better than any package manager ever can.
- goodpoint 5y ago

Tor3 5y ago

I understand the problem dpkg is facing (can't know the canonical pathname for an object is, in some cases), but I have huge problems understanding why a symlink farm is preferable. The last thing I want to see on my Debian system is directories with hundreds or maybe thousands of symlink entries. There must be a way to let dpkg do its job if the directory-symlink approach is used instead, surely? (/bin -> /usr/bin)

torstenvl 5y ago
If we're linking hundreds of individual files on a single large partition, why not make them hard links and avoid the clutter?
- rollcat 5y ago
  Dealing with hard links in the real world is a bit tricky. For example, "mv a b" is very different from "cat a > b". You have to do the equivalent of the latter in order to update a file in-place without updating every single hard link, which is problematic since writing to a file is not atomic the way an unlink+rename is. It's a bit of extra logic for the package manager to deal with, while a symlink is just a symlink.
- s28l 5y ago
  It's entirely possible that /usr might be a separate partition from /, so hardlinks wouldn't work.
codedokode 5y ago
If you link /bin to /usr/bin then all binaries will be available under /bin, and developers will hardcode paths like /bin/python3. This will make programs non-portable to other distributiobns with separate /bin and /usr/bin. The link farm doesn't have this problem.
- dTP90pN 5y ago
  Did that end up becoming a problem in all the other distros using aliased directories, for example Fedora which has 10 years of experience with this scheme? An quick look on Fedora 35 with $ grep -rP "(?!(?=(?'a'[\s\S]*))(?'b'\/usr(?=\k'a'\z)|(?<=(?=x^|(?&b))[\s\S])))\/bin\/(?!(sh|ksh|zsh|bsh|bash|dash|mount|umount|echo|true|false|ls|rm|mv|ln|cat))" /usr/bin doesn't really show any objectionable binaries being referred to via /bin (I count those I excluded in the negative lookahead as certainly non-objectionable). (I used the variable-length negative lookbehind [1] because I wanted to reduce false positives, but just scrolling through the results ended up being easier) [1] http://www.drregex.com/2019/02/variable-length-lookbehinds-actually.html http://www.drregex.com/2019/02/variable-length-lookbehinds-a... Alternatively, a simple grep --binary-files=text -rP '(?<!/usr)/bin/(?!sh|ksh|zsh|bsh|bash|dash|mount|umount|echo|true|false|ls|rm|mv|ln|cat)' /usr/bin to also include binary files does the job as well.

enriquto 5y ago

I'm surprised that they are pushing to remove /bin and keep /usr/bin. Why not the other way round? Let's keep our binaries out of /usr! (as a first step to remove /usr altogether, since it has a confusing name). I long for a day where PATH=/bin is all we need.

_wldu 5y ago
$HOME/bin is useful as well. I like that I can have my own software (that I wrote and compiled) in my home folder and don't need to ask a sys admin to install it for me.
- CameronNemo 5y ago
  I use ~/.local/bin, but yeah using your homedir for as much as possible makes system administration easier for everyone.
hashhar 5y ago
Not everything is a binary. Also having everything system specific live under a single path means easy backups, easy immutable trees that can be A/B swapped for seamless and online updates with easy rollbacks. No more wondering where the various tools live since everything lives in /usr/bin and more.
- zozbot234 5y ago
  The original point of a non-merged /bin (and /sbin) was to contain essential binaries for system recovery and early boot, that should be available even when a separate /usr/ partition is not yet mounted. Keep in mind that the approach of booting from initrd is quite non-idiomatic and used only for convenience wrt. a generic distro install; you're generally meant to recompile your kernel so that it can mount / directly at boot and find everything it needs there, without requiring to mount either a ramdisk or a separate partition. This means that early-boot stuff must live there in order to get the benefits of separate partitions in the first place.
  - 5y ago

isr 5y ago

Sigh ... gobolinux solved this quite elegantly over 20 years ago, and to this day most folks don't quite grok how simple and elegant their solution was.

smcl 5y ago
What was the solution, and why is is simple and elegant?
- chalst 5y ago
  There's a rather old review on LWN: https://lwn.net/Articles/66290/ https://lwn.net/Articles/66290/ It's essentially like Macos's /Applications directory, except where Macos supports the traditional POSIX fhs via a hidden /private root directory, Gobolinux hides these directories with a filesystem kludge.
  - smcl 5y ago
    Nice, thanks! I'll give this a little read over lunch :)
    - 0des 5y ago
      I recently did something similar refactoring XDG endpoints to get rid of the standard home layout. I was quite surprised everything just worked, and there wasn't an issue of directories being recreated in their original locations or file templates being missing. Quite liberating if you ask me.
Saint_Genet 5y ago
Having potentially hundreds of copies of popular shared libraries on disk isn't particularly elegant when you find a bug in those libraries
- rnhmjoj 5y ago
  I'm sure how you're getting to this conclusion. GoboLinux filesystem hierarchy just separates executables and shared objects by name and version [1]; unlike the standard FHS where everything goes into /usr/lib, /usr/bin, etc. For backward compatibility there are symlinks (that takes no disk space) to the standard directories, but these are hidden by the kernel so they don't show up and clutter your file manager. [1]: https://gobolinux.org/at_a_glance.html https://gobolinux.org/at_a_glance.html [2]: https://gobolinux.org/doc/articles/gobohide.html https://gobolinux.org/doc/articles/gobohide.html

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

iso1210 5y ago

> The /usr merge idea was first raised in "The Case for the /usr Merge" by Lennart Poettering in 2012. It came out of the systemd community Well that's a recipe for a long drawnout battle

ungamedplayer 5y ago
It is just people with emotional damage.
jillesvangurp 5y ago
As somebody who really does not care either way, all this just sounds more like moving problems rather than solving a problems. At least, I don't think there are many Linux systems where you can safely remove /bin from the path just yet. I just checked. My Manjaro install has a /bin and it is full of files that look like I would not want to lose them. So, they fixed the /bin to /usr/bin thing for some packages but clearly not all packages in Arch. I bet/blindly assume that's true in Fedora as well. I've always struggled with figuring out where stuff lives on the file system in different linux and unix derivatives. It's never where you expect/want it to be. I've used solaris, hp ux, mac os, and linux over the years. They all have similarly named directories, mostly for weird historic reasons. Is it /var/usr/god/knows/what or in /opt/foo/bar or in /usr/var/lib or /usr/share/lib. What about /etc, /usr/etc, /usr/local/etc? You can take almost any 1,2,3, and 4 symbol permutations of share, lib, var, bin, opt, and sbin and probably find something that expects files to exist under that path. Reducing the number of permutations would probably be helpful. People seem to just roll the dice and create some place where shit lives based on mostly just vague intentions, rules, heuristics, and interpretations of those associated with long dead unix variants from the nineteen eighties, weird naming conventions, and what not. So, what's the rule here? Some 'special' packages should install to /bin and some other not 'special' packages should install to /usr/bin? Why? Is there any agreement about what constitutes a 'special' package? Is there a good functional reason for having both directories? And then the whole bin/sbin distinction is kind of arbitrary as well. Some binaries are statically linked, others are not and require libraries to exist in yet more shared directories on a library path. It all boils down to users requiring a PATH variables (and, inevitably, a lot of other/similarly named variables). And of course the order of paths is also super relevant and kind of the point. So you look in /usr/bin before or after you look in /sbin? It's all a house of cards. Brittle by design and convention. The notion of taking a package and then fragmenting it over a multitude of shared directories is the problem that needs fixing. The role of a package maintainer is bridging those different notions of where stuff should live between different distributions with some convoluted scripts. It's a job that should not need doing and code that should not need to be written. The main differences between linux distributions boil down how none of them actually having solved this problem that ended up doing only loosely similar but clearly different things for mostly obscure reasons. They don't agree on where stuff should live, how it should be moved/copied/linked there, where and how things are configured, etc. Most of these differences are kind of arbitrary and petty. /sbin, /usr/bin, /usr/sbin, /bin, /usr/local/bin, etc. who cares? I pretty much need all of these in my path for things to work as intended. I don't see a good reason for more than 1 of these to exist. Apple kind of got this half right with the notion of mounting a package rather than installing it. Most applications install/uninstall via drag & drop. I always thought that was a neat idea. Of course, they then made it complicated by having /User/<uid>/Library/* and /Library/* directories anyway. So, most applications leave a lot of clutter there after you drag them to the trash-can. And they also have the usual contingent of unixy directories. And package managers like brew, macports, fink, and whatnot that sort of carved out different places in the filesystem where their stuff lives. So, I wouldn't go as far as saying that Apple solved the problem. But it does look like progress to me. Mounting stuff rather than fragmenting it all over the place is progress. Docker does this. And so do Flatpak and Snap. Flatpak and Snap are kind of tedious in their own way (e.g. opencl support is a PITA with Darktable and other packages that need that). But at least I have some stuff that I installed with those that actually works without having to be customized for every linux distribution.

richardfey 5y ago

Instead of all this drama the person that cannot see any way forward, despite a valid example of Ubuntu making this work, should be removed from the critical path by the instrument of a fork. Otherwise it will just be a matter of time before another bike-shedding, multiple years-spanning incident happens again. By creating a fork and proving that it works, the discussion becomes about whether to merge patches or adopt the fork rather than sterile "what if"s. Edit: added last paragraph

Zardoz84 5y ago

Giving that on Ubuntu this was done (and in other distros), and there isn't any issue ( I just noticed that I have a merged /usr for a long time), the dpkg maintainer it's being a jerk.

josefx 5y ago
As far as I can find there are issues, namely that installing a package with dpkg can leave the system in a bad state. If I understand it correctly dpkg normally prevents by checking the paths it modifies, which ends up non trivial if the paths are aliased. > the dpkg maintainer it's being a jerk. He is required to accept working patches, priority by the committee seems to be the removal of warnings about the broken support instead of actually fixing it.
- Beltalowda 5y ago
  > He is required to accept working patches The thing is that no one wants to interact with a maintainer hostile to the very idea of your patch. Sure, they may be "required" to accept it, but it's not going to be a fun process for anyone involved. Part of Boccassi's concern with their patch is that it might not be merged at all due to what they perceive as "moving goalposts" and "excuses". I think that's not entirely an unreasonable concern, and no one likes to work on patches that will never get merged. I don't know what the path forward for Debian would be here. Things seem ... difficult. Getting some of the social tension solved and having people "kiss and make up" would probably be a good start, but that can only work if people are willing.
  - josefx 5y ago
    > The thing is that no one wants to interact with a maintainer hostile to the very idea His hostility seems to be based on actual reasons if it is true that other systems that performed the unification basically throw out any guarantees made by dpkg. Providing a patch that fixes that would get rid of those reasons. > Part of Boccassi's concern with their patch is that it might not be merged at all due to what they perceive as "moving goalposts" and "excuses". Bocassi is also the guy who claimed the first incomplete patch was rejected without further comment, which going by the article wasn't the case. So his opinion of the maintainer is at best misinformed, at worst intentionally deceptive. Also going by the article a merge can be forced by the committee. So this seems to be a non issue.

nickcw 5y ago

Just in case anyone is wondering why we had (or used to have) /bin and /sbin directories as well as /usr/bin /usr/sbin here is my understanding of it. It is because on old Unix systems there frequently wasn't enough space to store /usr on the root disk. Therefore /usr was a separate disk which might not be available during boot. This meant that you needed to put all the binaries you needed for boot in /bin and /sbin because /usr/bin and /usr/sbin might not be mounted yet. This no longer makes sense in the days of large disks which fit /usr just fine and initrds which can mount all the disks before kicking off the things which might need those binaries. In fact I think the initrd has taken the place of /bin and /sbin on a modern system containing copies of all the binaries needed to mount the disks (like mount and fsck).

ntauthority 5y ago
Adding to this historical reasoning, it is to be noted that in traditional Unix, /usr was pretty much the equivalent of /home in Linux systems: the parent directory for user home directories, i.e. 'usr'. The existence of /usr/bin can probably be explained by the reasoning that, just like nowadays Linux systems often have a really big /home partition, early Unix systems eventually ran out of space for whatever volume would hold /bin, and with 'reinstalling' not remotely as easy an option as it would be these days, storing an extra 'bin' etc. directory in the 'larger partition for user directories' became commonplace.
- ink_13 5y ago
  Careful, you'll attract computing historians who will want to tell you that "usr" actually stands for "UNIX system resources"
  - Cockbrand 5y ago
    That's how I've learned it around 1995, and that the division between /[s]bin and /usr/[s]bin has the reason that the former are required for system boot, while the latter are mounted via NFS from a central server - why waste local disk space when these would be identical for every system anyway (in a homogenous environment, at least). While that made a lot of sense to me back then, I've become aware that it might not be absolutely accurate, though.

chx 5y ago

deleted 5y ago
[deleted]
dbrgn 5y ago
If you'd actually read the article, you'd see that the problem isn't merged /usr. Other distros moved to merged /usr just fine. Also, the decision to move Debian to merged /usr wasn't done by Poettering, it was done by the Debian technical committee. So please stop spreading such ad-hominem nonsense.
0des 5y ago

egberts1 5y ago

We can now kiss-gone the security for embedded world and its IoT-related security brethren. It is often the directory separation of /bin and /usr/bin that is used to denote the extent of firmware’s scope between these binaries that are require for a boot up and what are they require to support their applications. (Thanks, PDP-11). Such wonderful boundaries of IoT upgrade scope can and is often denoted as two-stage upgrade with /bin as read-only; now, not so much. This is yet another case of IoT system engineering design getting steamrolled by the diminutive desktop-mindsets despite progress by its systemd author. Try yanking that Ethernet cable from its physical socket and watch if your long-running deeply-stateful process can survive. (It won’t, unless you retrain it against KISS principle to ignore such netdev disappearance.)

yjftsjthsd-h 5y ago
> We can now kiss-gone the security for embedded world and its IoT-related security brethren. Is anyone using Debian for that kind of thing? When I hear embedded, I expect buildroot or at least Alpine Linux.
WhyNotHugo 5y ago
Using `/usr/bin/` readonly is also a thing on some few distros (e.g.: Fedora Silverblue), so maybe you want to consider that for your usecase. For device-local binaries (e.g.: not part of firmware), /usr/local/bin sounds like the right choice (also, somewhat in line what some BSDs do).
- egberts1 5y ago
  This is just another direction to desktop-hive mentality with a focus toward minimizing security. Perhaps, it might be ideal and suitable to Windows-ize for your case; the security modeling of many designs, not so much.
  - WhyNotHugo 5y ago
    How does this minimise security? I don't see how Window is relevant; it doesn't have /usr nor /usr/bin.

upofadown 5y ago

>Improved compatibility with other Unixes/Linuxes... A quick check reveals that OpenBSD presently has separate /bin and /sbin. FreeBSD also has a separate /lib. Exactly what were the other Unixes/Linuxes that this was supposed to improve compatibility with? I hope was not just Solaris...

faho 5y ago
The idea is, if something hardcodes e.g. /bin/grep because it was written on a system where that's where grep is, it will suddenly start working because now grep is also accessible via /bin/grep. So it's compatibility in the sense of making things work (instead of failing over pointless differences), not in the sense of being the same.
Beltalowda 5y ago
OpenBSD also has /usr/X11R6, where it puts the X stuff (their X distro is named Xenocara). This is another piece of legacy from back when you might want to have both /usr/X11R2 and /usr/X11R3. These days the X11R6 directory actually contains X11R7 shrug. FreeBSD used to do this too, but got rid of it in FreeBSD 6 (I think? Maybe 7? About ten years ago). Both systems install packages to /usr/local.
- tremon 5y ago
  Both systems install packages to /usr/local That's because "packages" in the BSD world (at least openbsd) are considered third-party options, the base system is considered static. Linux distributions don't really have the notion of a static base system with "package" addons, the distribution packages are the system.

solcloud 5y ago

You can't teach an old dog new tricks. Debian docker image has open issue for this for years too.

BlinkenBlinken 5y ago

Current state of the UNIX file hierarchy is a mess. Many incarnations of GNU/Linux systems didn't contribute to a more elegant yet simplified directory structure. Just pick the one you like the most and serves the purpose of your desires. As for merged /usr, recently using Arch Linux for desktop use. Works decent enough to getting things done.

tenebrisalietum 5y ago
Elaborate. I don't find the UNIX file hierarchy complicated, but there are some historical quirks and artifacts (like `/var/lib`). The difference between `/usr/X` and `/X` is about the worst of it IMHO. Everything important is usually at most 3 levels deep and the short names are very convenient (versus "C:\Users\xxx\Documents and Settings"). Compare to things like Windows: c:\windows\system32 that has 64 bit stuff in it, the whole c:\windows\syswow64 b.s., c:\Program Files (x86) and c:\Program Files, the hierarchy in the Windows registry which has "Windows", "Windows NT", "Microsoft Windows" (probably), etc.
- jeroenhd 5y ago
  All file system hierarchies contain complicated rules and systems that have been built in by the people who designed them decades ago. I still don't know what I'm supposed to do with /usr/local, what the difference is between /usr/share and /usr/local/share, what the point of /opt is if programs install their files and dependencies in /usr(/local?)/lib anyway and why I have /usr/lib, /usr/lib32 and /usr/lib64 when only two directories and the right environment variables should suffice. There are subfolders in /dev that feel like they don't need to be subfolders. There's /tmp and three other places that contain temporary files that should get cleaned on boot. PID files appear strwen across /var/run and /tmp/<magic directory name>. /var can contain just about everything. Most of /var feels like it should actually be inside /var/spool but you're not going to see much in there except for a mail queue to nowhere on desktop Linux machines. Then we come to the XDG standard everybody just blatantly ignores that tries to bring order to the chaos that is program-generated files in the home directory. And then there's also snap. Snap looks at any directory convention, laughs, spits in your face for good measure, and creates a folder called "snap" wherever the fuck it wants to. I'd purge it from my system if the snap people hadn't convinced some tools I use to support it as the main distribution method. I'm sure there are guides out there that explain every directory and their purpose. I've read one of those guides, noticed that at least a third of the common programs I use clearly haven't read it, and forgotten the details already. The file hierarchy of a fully-fledged desktop Linux is kind of a mess, and that's just what you get when your core system is formed by combining the work of hundreds or thousands of volunteer projects.

BlinkenBlinken 5y ago

Decabytes 5y ago

I wish most *Unixes would go the Gobo Linux route with a Program, Users, System, Data, Mount file hierarchy that is just symlinks under the hood to the old hierarchy

zokula 5y ago
CameronNemo 5y ago
Why rename? How does that benefit us? The only real difference gobo offers is separating program and system files, but I'm not even sure what the difference is there. Other than that: * /Users == /home * /Data == /var * /Mount == /mnt (?) so what do we get by breaking the world?
- Decabytes 5y ago
  > so what do we get by breaking the world? In the the gobo documentation^1 section called "But what about Unix compatibility?" > The GoboLinux system layout seems to be a major departure from the Unix tradition. Does this mean all programs need to adjusted so that they work with the new layout? Fortunately, the answer is no. Through a mapping of traditional paths into their GoboLinux counterparts, we transparently retain compatibility with the Unix legacy. ~] ls -l /dev/null | cut -b 45- /dev/null ~] ls -l /bin/sh | cut -b 45- sh -> /Programs/Bash/4.4/bin/bash ~] ls -l /usr/include/stdio.h | cut -b 45- stdio.h -> /Programs/Glibc/2.24/include/stdio.h > There is no rocket science to this: /bin is a link to /System/Index/bin. And as a matter of fact, so is /usr/bin. And /usr/sbin... all "binaries" directories map to the same place. Amusingly, this makes us even more compatible than some more standard-looking distributions. In GoboLinux, all standard paths work for all files, while other distros may struggle with incompatibilites such as scripts breaking when they refer to /usr/bin/foo when the file is actually in /usr/local/bin/foo. > You may have noticed that the Unix paths did not show up in the system root listing in the very first example. They are actually there, but they are concealed from view using the GoboHide kernel extension. This is for aesthetic purposes only and purely optional, though: GoboLinux does not require modifications in the kernel or any other system components. But our users seem to like it a lot. :-) So it doesn't break the world. The faq^2 also indicates this wasn't a change to make Linux more newbie friendly, but in my own words I think it does. 1. https://gobolinux.org/at_a_glance.html https://gobolinux.org/at_a_glance.html 2. https://gobolinux.org/faq.html https://gobolinux.org/faq.html

zh3 5y ago

I've always liked the /usr distinction, with systems set up so they will at least boot without /usr mounted (at an absolutely minimum, a statically linked /bin/sh is enough to get a command prompt). This is particularly valuable for setups where a large number of PCs can mount /usr read-only from a remote server (we run quite a number of diskless x86 amd ARM systems like this).

rektide 5y ago
Bind mount your small system's /usr to /lower, mount your bigger /upper, and use overlayfs to combine them. Looking forward ro some blog posts discussing how to support this use case; it's a good one. I think usr-merge is so much more managable though that it's totally worth doing.
- zh3 5y ago
  Must admit I'm surprised there don't seem to be more users out there doing exactly this. Makes installing a new program for everyone instant (put it in /usr, now available to all) while allowing personal customisations with the ability to reset the system to a pristine state in a second. Esp. combined with diskless, the computer hardware becomes irrelevant/disposable - just grab a fresh one, boot the same image and carry on (or boot your personal image at a hot desk).
CameronNemo 5y ago
Can you not use an initramfs to support mounting an ro NFS root? You could even ship the initramfs and kernel using tftp. Then the diskless systems just need enough storage for a bootloader like uboot or tianocore.

mmh0000 5y ago

The Filesystem Hierarchy Standard[0][1] is used by most Linux distributions as the authoritative source and description of the various system directories. As other commentators have already noted, the hierarchy on UNIX system has always been a bit of a mess, the FHS is an attempt to clean up while maintaining backwards compatibility. [0] https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html [1] https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard#History https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard#...

xyzzy_plugh 5y ago

As a Debian user in a previous life, and a current NixOS user, this whole debacle amuses me greatly, as I no longer give a shit about paths like /usr/bin or /bin, beyond a handful of binaries needed to bootstrap my environment. I've even moved those paths into a read-only filesystem on some systems!

jeltz 5y ago
This could happen in any distribution which does not have a powerful dictator at the top. Almost nothing about this issue has to do with paths. It is one maintainer of a core package who obstructs the Debian project which is enabled by Debian's distributed nature with no powerful dictator at the top.
Ericson2314 5y ago
Yeah came here to say the same. For them, silly tradeoffs between uniformity and what might be considered ambient authority or separation of concerns. For us? Nahhh best of both worlds.

smm11 5y ago

I thought Apple sort of did this already? (I know, OS X is not Linux.)

bumblebritches5 5y ago

deleted 5y ago

[deleted]