4 ms·

Reminder Linux is too big (MLoCs, and megabytes of object code) for it to be anywhere near bug-free. As Linux runs with full privileges, this is a major archit

by snvzz 3y ago

Reminder Linux is too big (MLoCs, and megabytes of object code) for it to be anywhere near bug-free. As Linux runs with full privileges, this is a major architectural issue. We should be looking at deploying well-architected systems based on microkernels (such as seL4). Some examples of such systems include Fuchsia, Genode and Managarm.

bayindirh 3y ago
Yet these escalation CVEs are getting rarer and rarer, while production systems are generally hardened with SELinux and AppArmor out of the box, making practically exploiting them much harder. The gist, is it requires a local user to be able to run processes, and not to be bound by SELinux or AppArmor to succeed. It should be patched, but it's not Heartbleed. P.S.: I didn't want to open the box of performance issues of microkernels, yet somebody did, and yes, performance matters.
- codedokode 3y ago
  > P.S.: I didn't want to open the box of performance issues of microkernels Why are there performance issues? Are there any reasons for them except for context switch costs (which are only due to legacy CPU architectures that were optimized for outdated insecure OSes)? Is microkernel architecture somehow incompatible with good performance?
  - samus 3y ago
    Microkernel architecture pretty much requires that processes communicate via explicitly defined interfaces, usually message passing. And it becomes much more difficult for subsystems to coordinate low-level activities, which is crucial for performance optimizations. And yes, context switches matter a lot too.
    - snvzz 3y ago
      Lots of myths around microkernel "slowness"[0]. >And yes, context switches matter a lot too. seL4 is significantly (100x-1000x) faster than Linux at context switching[1]. At the same time, it does not require that many more context switches. 0. https://archive.is/30hhi https://archive.is/30hhi 1. https://sel4.systems/About/Performance/ https://sel4.systems/About/Performance/
      - kramerger 3y ago
        This is a very unfair comparison 1. A sel4 context switch does significantly less than a Linux ditto. It saves fewer hardware registers etc 2. You will need many more context switches to achieve the same thing in a microkernel due to its distributed architecture. 3. A lot of "complexity" in the kernel is simply performance optimization that does not exist in sel4. Back in the days Minix and Hurd promised to achieve optimization by replacing generic servers with custom ones but I haven't seen this actually working in 20 years
- codedokode 3y ago
  > while production systems are generally hardened with SELinux and AppArmor out of the box, making practically exploiting them much harder. Is it really so? I get an impression that SELinux is a so complicated that nobody wants to bother with setting it up and everyone disables it as soon as possible.
  - mikaraento 3y ago
    You are correct about SELinux in general terms. However, there is a large scale exception to this: Android. Android uses SELinux extensively, and it is also used as part of further sandboxes in Chrome and Android System Intelligence.
  - bayindirh 3y ago
    We don't disable SELinux unless the software explicitly requires it. Also, all of the machines running without SELinux are not accessible from outside. SELinux is complicated, but it's not impossible to work with. Configuring new rules is another story, though.
  - gizmo686 3y ago
    RHEL systems run SELinux in enforcing mode by default. Most applications run in a domain called "unconfined_t", which is about as secure as it sounds. [0] Ubuntu runs AppArmor in its version of enforcing mode. I've only ever run into issues with it once though, so I presume it is a similarly lax policy. [0] The policy is called "targeted", to reflect the fact that you are supposed to write targeted policy modules for specific applications you are concerned about.
    - bayindirh 3y ago
      Debian’s AppArmor policy is pretty strict, yet well refined. As long as you use standard folders and paths, it’s pretty invisible. Last time while I was migrating a DNS server, our zone folders were not in a standard path, and Bind was unable to find them. 10 minutes later I remembered about AppArmor, and moved the files. The only problematic hardening was Thunderbird, which broke attachments. They disabled the Thunderbird’s profile after failing to fix it. Currently I’m on mobile, so I can’t give you a list of active profiles, yet the list is not short and the policy is not lax. I worked with both of them back in the day and developed profiles for them. Update: Stats are as follows: 78 packages ship AppArmor profiles in Debian 12. Notable are: - mariadb-server - redshift (f.lux alternative) - bind - ioquake3 (yes, the game) - squid - tcpdump among others. Also there are "apparmor-profiles" and "apparmor-profiles-extra" packages which provides profiles for vsftpd, dovecot, dhcpd amongst others (141 in profiles and 10 in extras). So, AppArmor ecosystem is neither lax, nor unmaintained.
- gizmo686 3y ago
  I don't see how SELinux or AppArmor would help here. From the details provided, it looks like all that is needed is mmap. At most it "requires minimal capabilities to trigger" based on the article. Granted, it is possible to deny mmap permissions, but I don’t think most policy writers would consider mmap a particular dangerous permission to grant. I just checked my CentOS 8 system running stock policy, and every process type has permission to map every file type. Granted, this is gated behind a boolean, so an administrator could disable it if they wanted to, but the default is true. Not that it is relevant to this issue where it doesn't matter what you mmap; but since I know someone will ask, having mmap permission does not bypass the check on, for example, read permission.
  - blibble 3y ago
    you "disable" mmap and you will break: - dynamic loading - malloc - any just in time compilers it is a core feature of a modern operating system
- tptacek 3y ago
  It's worse than Heartbleed for anyone doing shared-kernel multitenant (running Docker containers from mutually untrusting sources). There are a bunch of those!
  - bayindirh 3y ago
    If you're running multi-tenant Docker, you're doing something wrong to begin with. There are better solutions like Apptainer [0]. [0]: https://apptainer.org/ https://apptainer.org/
    - tptacek 3y ago
      I don't know what this is, but it appears to be shared-kernel.
    - IshKebab 3y ago
      Apptainer works the same way as Docker. It just makes using it for distributing applications more convenient.
  - hollerith 3y ago
    Does browsing the web with Javascript turned on on the same kernel where I ssh from (and consequently keep my ssh credentials) count as multitenanted here?
mymac 3y ago
Yes, but during that debate ages ago Linus won on points and as a result we now have an OS focused on throughput rather than on security.
- surajrmal 3y ago
  Not every device using Linux is a server. Many devices can deal with hypothetical peak throughput losses for the sake of improved security. Power and latency are probably going the key metrics that need to not be regressed on, but even then, some devices probably still don't care as much about those.
- snvzz 3y ago
  Linus was never correct about microkernels[0], and the world is worse for it. 0. https://www.cosy.sbg.ac.at/~clausen/PVSE2006/linus-rebuttal.html https://www.cosy.sbg.ac.at/~clausen/PVSE2006/linus-rebuttal....
Zamiel_Snawley 3y ago
While I agree in principle, I have tried a couple times(admittedly not very hard) to learn how to use seL4 and it seems unready for prime time. I don't think POSIX is the end-all-be-all, but it is what I and many others are familiar with, and seL4 has practically nothing in common with it. The design is just too foreign to get going quickly, and the opportunity cost of learning their way to do things is too high in almost all cases. One day I hope to have the time to dedicate to learning it properly!
- pjmlp 3y ago
  POSIX also hardly matters on Windows, mobile OSes, legacy mainframes/micros or game consoles. What is needed is to have a product that makes the OS relevant enough for devs to consider targeting the platform.
  - surajrmal 3y ago
    The problem is that sel4 isn't turnkey enough for the average product maker to use. It's good for larger firms who can invest in building out their own platform on top which they can then use in products. There doesn't seem to be anyone building a public platform on top of sel4 which can be then used by product makers. This is similar to building a product with a custom Linux distro vs an existing one, albeit a custom Linux distro is still far easier to do because of the great ecosystem around it. I'm not an advocate for using Linux, but unless someone invests in the missing pieces, sel4 usage will never become widespread.
    - snvzz 3y ago
      > unless someone invests in the missing pieces, sel4 usage will never become widespread. There's multiple efforts, such as Makatea[0], Genode[1] and sel4cp[2]. The seL4 foundation is finally well-established and funded thanks to increasing commercial interest[3]. There's been a lot of activity lately[4] and I'm very optimistic for it. 0. https://trustworthy.systems/projects/TS/makatea https://trustworthy.systems/projects/TS/makatea 1. https://www.genode.org/ https://www.genode.org/ 2. https://trustworthy.systems/projects/TS/sel4cp/ https://trustworthy.systems/projects/TS/sel4cp/ 3. https://sel4.systems/Foundation/Membership/ https://sel4.systems/Foundation/Membership/ 4. https://trustworthy.systems/news/ https://trustworthy.systems/news/
deleted 3y ago
[deleted]
zokier 3y ago
Most of those millions of lines of code are drivers which are largely irrelevant factor here. The actual core Linux is, while still large, significantly smaller and tighter codebase
saghm 3y ago
> Reminder Linux is too big (MLoCs, and megabytes of object code) for it to be anywhere near bug-free. I don't necessarily disagree, but I also think that almost every piece of software in common use is probably too big to actually be bug free. The bar is far lower than millions of lines of code for "to big to not have bugs"; I'd expect that line to be crossed in the tens of thousands, if not lower.

by snvzz 3y ago

bayindirh 3y ago

Yet these escalation CVEs are getting rarer and rarer, while production systems are generally hardened with SELinux and AppArmor out of the box, making practically exploiting them much harder. The gist, is it requires a local user to be able to run processes, and not to be bound by SELinux or AppArmor to succeed. It should be patched, but it's not Heartbleed. P.S.: I didn't want to open the box of performance issues of microkernels, yet somebody did, and yes, performance matters.

codedokode 3y ago
> P.S.: I didn't want to open the box of performance issues of microkernels Why are there performance issues? Are there any reasons for them except for context switch costs (which are only due to legacy CPU architectures that were optimized for outdated insecure OSes)? Is microkernel architecture somehow incompatible with good performance?
- samus 3y ago
  Microkernel architecture pretty much requires that processes communicate via explicitly defined interfaces, usually message passing. And it becomes much more difficult for subsystems to coordinate low-level activities, which is crucial for performance optimizations. And yes, context switches matter a lot too.
  - snvzz 3y ago
    Lots of myths around microkernel "slowness"[0]. >And yes, context switches matter a lot too. seL4 is significantly (100x-1000x) faster than Linux at context switching[1]. At the same time, it does not require that many more context switches. 0. https://archive.is/30hhi https://archive.is/30hhi 1. https://sel4.systems/About/Performance/ https://sel4.systems/About/Performance/
    - kramerger 3y ago
      This is a very unfair comparison 1. A sel4 context switch does significantly less than a Linux ditto. It saves fewer hardware registers etc 2. You will need many more context switches to achieve the same thing in a microkernel due to its distributed architecture. 3. A lot of "complexity" in the kernel is simply performance optimization that does not exist in sel4. Back in the days Minix and Hurd promised to achieve optimization by replacing generic servers with custom ones but I haven't seen this actually working in 20 years
codedokode 3y ago
> while production systems are generally hardened with SELinux and AppArmor out of the box, making practically exploiting them much harder. Is it really so? I get an impression that SELinux is a so complicated that nobody wants to bother with setting it up and everyone disables it as soon as possible.
- mikaraento 3y ago
  You are correct about SELinux in general terms. However, there is a large scale exception to this: Android. Android uses SELinux extensively, and it is also used as part of further sandboxes in Chrome and Android System Intelligence.
- bayindirh 3y ago
  We don't disable SELinux unless the software explicitly requires it. Also, all of the machines running without SELinux are not accessible from outside. SELinux is complicated, but it's not impossible to work with. Configuring new rules is another story, though.
- gizmo686 3y ago
  RHEL systems run SELinux in enforcing mode by default. Most applications run in a domain called "unconfined_t", which is about as secure as it sounds. [0] Ubuntu runs AppArmor in its version of enforcing mode. I've only ever run into issues with it once though, so I presume it is a similarly lax policy. [0] The policy is called "targeted", to reflect the fact that you are supposed to write targeted policy modules for specific applications you are concerned about.
  - bayindirh 3y ago
    Debian’s AppArmor policy is pretty strict, yet well refined. As long as you use standard folders and paths, it’s pretty invisible. Last time while I was migrating a DNS server, our zone folders were not in a standard path, and Bind was unable to find them. 10 minutes later I remembered about AppArmor, and moved the files. The only problematic hardening was Thunderbird, which broke attachments. They disabled the Thunderbird’s profile after failing to fix it. Currently I’m on mobile, so I can’t give you a list of active profiles, yet the list is not short and the policy is not lax. I worked with both of them back in the day and developed profiles for them. Update: Stats are as follows: 78 packages ship AppArmor profiles in Debian 12. Notable are: - mariadb-server - redshift (f.lux alternative) - bind - ioquake3 (yes, the game) - squid - tcpdump among others. Also there are "apparmor-profiles" and "apparmor-profiles-extra" packages which provides profiles for vsftpd, dovecot, dhcpd amongst others (141 in profiles and 10 in extras). So, AppArmor ecosystem is neither lax, nor unmaintained.
gizmo686 3y ago
I don't see how SELinux or AppArmor would help here. From the details provided, it looks like all that is needed is mmap. At most it "requires minimal capabilities to trigger" based on the article. Granted, it is possible to deny mmap permissions, but I don’t think most policy writers would consider mmap a particular dangerous permission to grant. I just checked my CentOS 8 system running stock policy, and every process type has permission to map every file type. Granted, this is gated behind a boolean, so an administrator could disable it if they wanted to, but the default is true. Not that it is relevant to this issue where it doesn't matter what you mmap; but since I know someone will ask, having mmap permission does not bypass the check on, for example, read permission.
- blibble 3y ago
  you "disable" mmap and you will break: - dynamic loading - malloc - any just in time compilers it is a core feature of a modern operating system
tptacek 3y ago
It's worse than Heartbleed for anyone doing shared-kernel multitenant (running Docker containers from mutually untrusting sources). There are a bunch of those!
- bayindirh 3y ago
  If you're running multi-tenant Docker, you're doing something wrong to begin with. There are better solutions like Apptainer [0]. [0]: https://apptainer.org/ https://apptainer.org/
  - tptacek 3y ago
    I don't know what this is, but it appears to be shared-kernel.
  - IshKebab 3y ago
    Apptainer works the same way as Docker. It just makes using it for distributing applications more convenient.
- hollerith 3y ago
  Does browsing the web with Javascript turned on on the same kernel where I ssh from (and consequently keep my ssh credentials) count as multitenanted here?

mymac 3y ago

Yes, but during that debate ages ago Linus won on points and as a result we now have an OS focused on throughput rather than on security.

surajrmal 3y ago
Not every device using Linux is a server. Many devices can deal with hypothetical peak throughput losses for the sake of improved security. Power and latency are probably going the key metrics that need to not be regressed on, but even then, some devices probably still don't care as much about those.
snvzz 3y ago
Linus was never correct about microkernels[0], and the world is worse for it. 0. https://www.cosy.sbg.ac.at/~clausen/PVSE2006/linus-rebuttal.html https://www.cosy.sbg.ac.at/~clausen/PVSE2006/linus-rebuttal....

Zamiel_Snawley 3y ago

While I agree in principle, I have tried a couple times(admittedly not very hard) to learn how to use seL4 and it seems unready for prime time. I don't think POSIX is the end-all-be-all, but it is what I and many others are familiar with, and seL4 has practically nothing in common with it. The design is just too foreign to get going quickly, and the opportunity cost of learning their way to do things is too high in almost all cases. One day I hope to have the time to dedicate to learning it properly!

pjmlp 3y ago
POSIX also hardly matters on Windows, mobile OSes, legacy mainframes/micros or game consoles. What is needed is to have a product that makes the OS relevant enough for devs to consider targeting the platform.
- surajrmal 3y ago
  The problem is that sel4 isn't turnkey enough for the average product maker to use. It's good for larger firms who can invest in building out their own platform on top which they can then use in products. There doesn't seem to be anyone building a public platform on top of sel4 which can be then used by product makers. This is similar to building a product with a custom Linux distro vs an existing one, albeit a custom Linux distro is still far easier to do because of the great ecosystem around it. I'm not an advocate for using Linux, but unless someone invests in the missing pieces, sel4 usage will never become widespread.
  - snvzz 3y ago
    > unless someone invests in the missing pieces, sel4 usage will never become widespread. There's multiple efforts, such as Makatea[0], Genode[1] and sel4cp[2]. The seL4 foundation is finally well-established and funded thanks to increasing commercial interest[3]. There's been a lot of activity lately[4] and I'm very optimistic for it. 0. https://trustworthy.systems/projects/TS/makatea https://trustworthy.systems/projects/TS/makatea 1. https://www.genode.org/ https://www.genode.org/ 2. https://trustworthy.systems/projects/TS/sel4cp/ https://trustworthy.systems/projects/TS/sel4cp/ 3. https://sel4.systems/Foundation/Membership/ https://sel4.systems/Foundation/Membership/ 4. https://trustworthy.systems/news/ https://trustworthy.systems/news/

deleted 3y ago

[deleted]

zokier 3y ago

Most of those millions of lines of code are drivers which are largely irrelevant factor here. The actual core Linux is, while still large, significantly smaller and tighter codebase

saghm 3y ago

> Reminder Linux is too big (MLoCs, and megabytes of object code) for it to be anywhere near bug-free. I don't necessarily disagree, but I also think that almost every piece of software in common use is probably too big to actually be bug free. The bar is far lower than millions of lines of code for "to big to not have bugs"; I'd expect that line to be crossed in the tens of thousands, if not lower.