4 ms·

Revisiting how we put together Linux systems (2014)

55 points by mondoshawan 7y ago

detaro 7y ago
(2014)
mongol 7y ago
Were these ideas ever picked up?
- adisbladis 7y ago
  These specific ideas? To the best of my knowledge, no. Other related ideas like Nix/NixOS/Guix are solving some of the same problems outlined in the article in a much more systematic way that are not reliant on any one file system.
  - spindle 7y ago
    And also, if I understand correctly, NixOS and Guix need much less help from developers than this proposal would have. Specifically, NixOS and Guix don't care how anyone names anything (except, of course, their dogs).
    - DyslexicAtheist 7y ago
      is anyone using these systems in production? I would immediately apply to work in such a place. It's a catch-22. The biggest gain in using NixOS/Guix are production systems big enough where it matters. And every place that grows big enough where this would be good for the them they usually roll their own ... Because tech-debt doesn't allow them from switching to another OS + they have so much duct-tape that if they'd replace it, would also have to find work for those doing the duct-taping. I think it's not a technical problem at all, nor do I think it's because we lack ideas.
      - spindle 7y ago
        Yes, there are places that are using NixOS in production. I don't know the details, so this is a placeholder for someone else to fill in - or else ask on the NixOS subreddit or IRC channel.
    - the_why_of_y 7y ago
      Nix/Guix is a very simple and elegant design, but I find it has a downside: because every package is immutable, a security fix in a low-level shared library (worst case: glibc) requires recompiling every package that depends on it, and because there is no difference between OS and applications - everything is a Nix package - you have to wait until the whole dependency graph is recompiled before you can effectively install the security update. TFA's design avoids this by separating out both a base OS layer and an application runtime layer, both of which have a bounded size.
      - xfer 7y ago
        There is a hack to patch only essential component of a system without rebuilding all dependent pkgs : https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/replace-dependency.nix https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-supp... .
- pzmarzly 7y ago
  I think Flatpak came out of it. If you take a look at 2015 thread about it, you will find this comment: https://news.ycombinator.com/item?id=9477863 https://news.ycombinator.com/item?id=9477863 But I'm not aware of any OS built upon btrfs volumes. I never used btrfs, but I heard its performance degrades a lot once you get to more than 100 volumes (or was it 100 snapshots? I'm not sure). Finally, the proposed system wouldn't help that much with security. Malicious programs would still be able to send your .ssh dir over the Internet.
  - cmurf 7y ago
    SLES and openSUSE use Btrfs by default, including snapshots and rollbacks. Snapshot creation is inexpensive. Snapshot deletion can be expensive depending on the changes since snapshot was taken, but this is workload specific. All subvolumes/snapshots share the same extent tree, so that's where things can get expensive. This is from 2017, I'm sure there are other more recent anecdotes, and Btrfs performance has improved since then also, in particular qgroups. https://lore.kernel.org/linux-btrfs/CAMp4zn8YUdVShFibUKCXtwZTZpicCbmm7zSYMn7+K5CNt-cxGA@mail.gmail.com/ https://lore.kernel.org/linux-btrfs/CAMp4zn8YUdVShFibUKCXtwZ...
- the_why_of_y 7y ago
  The proposal talks a lot about using btrfs features, but there doesn't appear to be anything based on that. However if you look at the design goals, most of them have been implemented, although there is not yet wide usage. The best way to try this out in a runable desktop OS is probably Silverblue. https://silverblue.fedoraproject.org/ https://silverblue.fedoraproject.org/ > We want a unified scheme, how we can install and update OS images, user apps, runtimes and frameworks. The storage backend for this is OSTree, using HTTP as the transport; it's analogous to Git for binary files and does not require special filesystem features, only hardlinks. https://ostree.readthedocs.io/en/latest/ https://ostree.readthedocs.io/en/latest/ > We want a unified scheme how you can relatively freely mix OS images, apps, runtimes and frameworks on the same system. Flatpak provides the abstraction layer for this; the base OS can be anything and the applications use only libraries from a Flatpak runtime; by default, Flatpak uses OSTree to store applications. Snap is similar, although it requires AppArmor, so can't run on many popular distros, and I think there's only one runtime that you're required to use, so no "mix and match". Docker is similar, although it can't handle desktop applications in a sensible manner (no sandbox). > We want a fully trusted system, where cryptographic verification of all executed code can be done, all the way to the firmware, as standard feature of the system. There are several points here: * firmware/pre-boot: [Edit: TPM can be used to verify this, but it's not set up that way by default; Windows 10 Bitlocker can rely on TPM for this] * bootloader: UEFI Secure Boot handles this * kernel/initramfs: I'm afraid this isn't typically verified currently, see https://mjg59.dreamwidth.org/54203.html https://mjg59.dreamwidth.org/54203.html * base OS and Flatpak: OSTree can require GPG signatures for installation; to check the files at runtime, you'd need an additional layer such as dm-verity. > We want to allow app vendors to write their programs against very specific frameworks, under the knowledge that they will end up being executed with the exact same set of libraries chosen. FlatPak SDKs provide this. Snap has an SDK as well, although note the singular, Snap is exclusively controlled by one company. > We want to allow parallel installation of multiple OSes and versions of them, multiple runtimes in multiple versions, as well as multiple frameworks in multiple versions. And of course, multiple apps in multiple versions. Flatpak can do multiple framework versions by default, and multiple application versions with command line flags. Multiple different OSes hasn't happened yet, probably you'd want at least one non-experimental OS first... > We want everything double buffered (or actually n-fold buffered), to ensure we can reliably update/rollback versions, in particular to safely do automatic updates. OSTree does this. > We want a system where updating a runtime, OS, framework, or OS container is as simple as adding in a new snapshot and restarting the runtime/OS/framework/OS container. OSTree/Flatpak again. > We want a system where we can easily instantiate a number of OS instances from a single vendor tree, with zero difference for doing this on order to be able to boot it on bare metal/VM or as a container. No idea if this exists anywhere yet. > We want to enable Linux to have an open scheme that people can use to build app markets and similar schemes, not restricted to a specific vendor. Flatpak/FlatHub solve this; Snap does not because it's exclusively controlled by one vendor.
83hajfjeijg 7y ago
Just say no.
dang 7y ago
A thread from 2017: https://news.ycombinator.com/item?id=13782786 https://news.ycombinator.com/item?id=13782786 2015: https://news.ycombinator.com/item?id=9477612 https://news.ycombinator.com/item?id=9477612 Discussed at the time: https://news.ycombinator.com/item?id=8251288 https://news.ycombinator.com/item?id=8251288
- dijit 7y ago
  This is super interesting, reading the comments you can easily see the sentiment steadily getting less hostile to Poetterings ideas. I guess the only thing you need is total control of all major distros to change sentiment.
DyslexicAtheist 7y ago
> When a developer wants to develop against a specific runtime he installs the right framework, and then temporarily transitions into a name space where /usris mounted from the framework sub-volume, and /home/$USER from his own home directory. In this name space he then runs his build commands. He can build in multiple name spaces at the same time, if he intends to builds software for multiple runtimes or architectures at the same time. it reminds me of the Clearcase version control system. You would run `cleartool setview <viewname>` to enter the namespace of that "view". Like Las Vegas, everything you did within that view would stay in that view. This was before containers and VM's became widespread. It felt a like magic, because you could run one command and see a totally different build and tooling environment. Under the hood it meant that running `make` would pull in objects from other teams who used exactly the same view as you (so like ccache but across the network scaled to N-engineers). Most companies have thrown clearcase out when svn/git came along because poor administration practices could push license / storage costs through the roof (and it was never cheap to begin with). We have git/mercurial,... today which I'm quite happy for. But considering how old all these ideas are, it's a shame FOSS is still stuck in the stone age in this regard. [1] https://www.ibm.com/support/knowledgecenter/SSSH27_8.0.1/com.ibm.rational.clearcase.cc_ref.doc/topics/ct_setview.htm https://www.ibm.com/support/knowledgecenter/SSSH27_8.0.1/com...
- pjmlp 7y ago
  FOSS feels stuck in the stone age for many things, because doing something like ClearCase, world class IDEs or game middleware requires typical corporation level budgets, and since most don't want to pay for tooling, only corporate developers get the nice toys. Another thing that ClearCase was already great at, 20 years ago, was providing a file system cache for C and C++ builds, providing a mechanism to only compile translation units that had outdated object files across all developer team.
- pjc50 7y ago
  And yet ClearCase was notorious for being the bad old world that everyone wanted to get away from. I'm sure someone will be along with war stories to tell us why the reality wasn't so great.

55 points by mondoshawan 7y ago

detaro 7y ago

(2014)

mongol 7y ago

Were these ideas ever picked up?

adisbladis 7y ago
These specific ideas? To the best of my knowledge, no. Other related ideas like Nix/NixOS/Guix are solving some of the same problems outlined in the article in a much more systematic way that are not reliant on any one file system.
- spindle 7y ago
  And also, if I understand correctly, NixOS and Guix need much less help from developers than this proposal would have. Specifically, NixOS and Guix don't care how anyone names anything (except, of course, their dogs).
  - DyslexicAtheist 7y ago
    is anyone using these systems in production? I would immediately apply to work in such a place. It's a catch-22. The biggest gain in using NixOS/Guix are production systems big enough where it matters. And every place that grows big enough where this would be good for the them they usually roll their own ... Because tech-debt doesn't allow them from switching to another OS + they have so much duct-tape that if they'd replace it, would also have to find work for those doing the duct-taping. I think it's not a technical problem at all, nor do I think it's because we lack ideas.
    - spindle 7y ago
      Yes, there are places that are using NixOS in production. I don't know the details, so this is a placeholder for someone else to fill in - or else ask on the NixOS subreddit or IRC channel.
  - the_why_of_y 7y ago
    Nix/Guix is a very simple and elegant design, but I find it has a downside: because every package is immutable, a security fix in a low-level shared library (worst case: glibc) requires recompiling every package that depends on it, and because there is no difference between OS and applications - everything is a Nix package - you have to wait until the whole dependency graph is recompiled before you can effectively install the security update. TFA's design avoids this by separating out both a base OS layer and an application runtime layer, both of which have a bounded size.
    - xfer 7y ago
      There is a hack to patch only essential component of a system without rebuilding all dependent pkgs : https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/replace-dependency.nix https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-supp... .
pzmarzly 7y ago
I think Flatpak came out of it. If you take a look at 2015 thread about it, you will find this comment: https://news.ycombinator.com/item?id=9477863 https://news.ycombinator.com/item?id=9477863 But I'm not aware of any OS built upon btrfs volumes. I never used btrfs, but I heard its performance degrades a lot once you get to more than 100 volumes (or was it 100 snapshots? I'm not sure). Finally, the proposed system wouldn't help that much with security. Malicious programs would still be able to send your .ssh dir over the Internet.
- cmurf 7y ago
  SLES and openSUSE use Btrfs by default, including snapshots and rollbacks. Snapshot creation is inexpensive. Snapshot deletion can be expensive depending on the changes since snapshot was taken, but this is workload specific. All subvolumes/snapshots share the same extent tree, so that's where things can get expensive. This is from 2017, I'm sure there are other more recent anecdotes, and Btrfs performance has improved since then also, in particular qgroups. https://lore.kernel.org/linux-btrfs/CAMp4zn8YUdVShFibUKCXtwZTZpicCbmm7zSYMn7+K5CNt-cxGA@mail.gmail.com/ https://lore.kernel.org/linux-btrfs/CAMp4zn8YUdVShFibUKCXtwZ...
the_why_of_y 7y ago
The proposal talks a lot about using btrfs features, but there doesn't appear to be anything based on that. However if you look at the design goals, most of them have been implemented, although there is not yet wide usage. The best way to try this out in a runable desktop OS is probably Silverblue. https://silverblue.fedoraproject.org/ https://silverblue.fedoraproject.org/ > We want a unified scheme, how we can install and update OS images, user apps, runtimes and frameworks. The storage backend for this is OSTree, using HTTP as the transport; it's analogous to Git for binary files and does not require special filesystem features, only hardlinks. https://ostree.readthedocs.io/en/latest/ https://ostree.readthedocs.io/en/latest/ > We want a unified scheme how you can relatively freely mix OS images, apps, runtimes and frameworks on the same system. Flatpak provides the abstraction layer for this; the base OS can be anything and the applications use only libraries from a Flatpak runtime; by default, Flatpak uses OSTree to store applications. Snap is similar, although it requires AppArmor, so can't run on many popular distros, and I think there's only one runtime that you're required to use, so no "mix and match". Docker is similar, although it can't handle desktop applications in a sensible manner (no sandbox). > We want a fully trusted system, where cryptographic verification of all executed code can be done, all the way to the firmware, as standard feature of the system. There are several points here: * firmware/pre-boot: [Edit: TPM can be used to verify this, but it's not set up that way by default; Windows 10 Bitlocker can rely on TPM for this] * bootloader: UEFI Secure Boot handles this * kernel/initramfs: I'm afraid this isn't typically verified currently, see https://mjg59.dreamwidth.org/54203.html https://mjg59.dreamwidth.org/54203.html * base OS and Flatpak: OSTree can require GPG signatures for installation; to check the files at runtime, you'd need an additional layer such as dm-verity. > We want to allow app vendors to write their programs against very specific frameworks, under the knowledge that they will end up being executed with the exact same set of libraries chosen. FlatPak SDKs provide this. Snap has an SDK as well, although note the singular, Snap is exclusively controlled by one company. > We want to allow parallel installation of multiple OSes and versions of them, multiple runtimes in multiple versions, as well as multiple frameworks in multiple versions. And of course, multiple apps in multiple versions. Flatpak can do multiple framework versions by default, and multiple application versions with command line flags. Multiple different OSes hasn't happened yet, probably you'd want at least one non-experimental OS first... > We want everything double buffered (or actually n-fold buffered), to ensure we can reliably update/rollback versions, in particular to safely do automatic updates. OSTree does this. > We want a system where updating a runtime, OS, framework, or OS container is as simple as adding in a new snapshot and restarting the runtime/OS/framework/OS container. OSTree/Flatpak again. > We want a system where we can easily instantiate a number of OS instances from a single vendor tree, with zero difference for doing this on order to be able to boot it on bare metal/VM or as a container. No idea if this exists anywhere yet. > We want to enable Linux to have an open scheme that people can use to build app markets and similar schemes, not restricted to a specific vendor. Flatpak/FlatHub solve this; Snap does not because it's exclusively controlled by one vendor.

83hajfjeijg 7y ago

Just say no.

dang 7y ago

A thread from 2017: https://news.ycombinator.com/item?id=13782786 https://news.ycombinator.com/item?id=13782786 2015: https://news.ycombinator.com/item?id=9477612 https://news.ycombinator.com/item?id=9477612 Discussed at the time: https://news.ycombinator.com/item?id=8251288 https://news.ycombinator.com/item?id=8251288

dijit 7y ago
This is super interesting, reading the comments you can easily see the sentiment steadily getting less hostile to Poetterings ideas. I guess the only thing you need is total control of all major distros to change sentiment.

DyslexicAtheist 7y ago

> When a developer wants to develop against a specific runtime he installs the right framework, and then temporarily transitions into a name space where /usris mounted from the framework sub-volume, and /home/$USER from his own home directory. In this name space he then runs his build commands. He can build in multiple name spaces at the same time, if he intends to builds software for multiple runtimes or architectures at the same time. it reminds me of the Clearcase version control system. You would run `cleartool setview <viewname>` to enter the namespace of that "view". Like Las Vegas, everything you did within that view would stay in that view. This was before containers and VM's became widespread. It felt a like magic, because you could run one command and see a totally different build and tooling environment. Under the hood it meant that running `make` would pull in objects from other teams who used exactly the same view as you (so like ccache but across the network scaled to N-engineers). Most companies have thrown clearcase out when svn/git came along because poor administration practices could push license / storage costs through the roof (and it was never cheap to begin with). We have git/mercurial,... today which I'm quite happy for. But considering how old all these ideas are, it's a shame FOSS is still stuck in the stone age in this regard. [1] https://www.ibm.com/support/knowledgecenter/SSSH27_8.0.1/com.ibm.rational.clearcase.cc_ref.doc/topics/ct_setview.htm https://www.ibm.com/support/knowledgecenter/SSSH27_8.0.1/com...

pjmlp 7y ago
FOSS feels stuck in the stone age for many things, because doing something like ClearCase, world class IDEs or game middleware requires typical corporation level budgets, and since most don't want to pay for tooling, only corporate developers get the nice toys. Another thing that ClearCase was already great at, 20 years ago, was providing a file system cache for C and C++ builds, providing a mechanism to only compile translation units that had outdated object files across all developer team.
pjc50 7y ago
And yet ClearCase was notorious for being the bad old world that everyone wanted to get away from. I'm sure someone will be along with war stories to tell us why the reality wasn't so great.