7 ms·

Could you explain how SELinux could ever sandbox against RCE in sshd? Its purpose is to grant login shells to arbitrary users, after all.

ajross 3y ago
It's possible to spawn a sshd as an unprivileged or partially-capabilitized process. Such as sandbox isn't the default deployment, but it's done often enough and would work as designed to prevent privilege elevation above the sshd process.
- admax88qqq 3y ago
  How can sshd spawn interactive sessions for other users if it's sandboxed?
  - ajross 3y ago
    Plausibly by having set-user-ID capability but not others an attacker might need. But in the more common case it just doesn't: you have an sshd running on a dedicated port for the sole purpose of running some service or another under a specific sandboxed UID. That's basically the github business model, for example.
    - dotancohen 3y ago
      I need full filesystem access, VIM, ls, cd, grep, awk, df, du at the very least. Sometimes perl, find, ncdu, and other utilities are necessary as well. Are you suggesting that each tool have its own SSH process wrapping it? Maybe write a shell to coordinate between them? It should support piping and output redirection, please.
      - ajross 3y ago
        Sigh. I'm not saying there's a sandboxed sshd setup that has equivalent functionality to the default one in your distro. I'm not even saying that there's one appropriate for your app. I'm saying, as a response to the point above, that sandboxing sshd is absolutely a valid defense-in-depth technique for privilege isolation, that it would work against attacks like this one to prevent whole-system exploitation, and that it's very commonly deployed in practice (c.f. running a git/ssh server a-la github).
        semiquaver 3y ago
        Git’s use of the ssh protocol as a transport is a niche use case that ignores the actual problem. No one is seriously arguing that you can’t sandbox that constrained scenario but it’s not really relevant since it’s not the main purpose of the secure shell daemon.
        dotancohen 3y ago
        The focus on the first S is good, yes, but SSH has another S and an H that needs focus as well.
    - deleted 3y ago
      [deleted]
  - kbolino 3y ago
    SELinux does not rely on the usual UID/GID to determine what a process can do. System services, even when running as "root", are running as confined users in SELinux. Confined root cannot do anything which SELinux policy does not allow it to do. This means you can let sshd create new sessions for non-root users while still blocking it from doing the other things which unconfined root would be able to do. This is still a lot of power but it's not the godlike access which a person logged in as (unconfined) root has.
    - quotemstr 3y ago
      Doesn't matter. A malicious sshd able to run commands arbitrary users can just run malicious commands as those users. We'd need something more like a cryptographically attested setreuid() and execve() combination that would run only commands signed with the private key of the intended user. You'd want to use a shared clock or something to protect against replay attacks
      - kbolino 3y ago
        Yes, this won't directly protect against an attacker whose goal is to create a botnet, mine some crypto on your dime, etc. However, it will protect against corruption of the O/S itself and, in tandem with other controls, can limit the abilities an attacker has, and ensure things like auditing are still enforced (which can be tied to monitoring, and also used for forensics). Whether it's worth it or not depends on circumstances. In many cloud environments, nuking the VM instance and starting over is probably easier than fiddling with SELinux.
        CanaryLayout 3y ago
        even easier is to STOP HOSTING SSHD ON IPV4 ON CLEARNET at minimum, ipv6 only if you absolutely must do it (it absolutely cuts the scans way down) better is to only host it on vpn even better is to only activate it with a portknocker, over vpn even better-better is to set up a private ipv6 peer-to-peer cloud and socat/relay to the private ipv6 network (yggdrasil comes to mind, but there's other solutions to darknet) your sshd you need for server maintenance/scp/git/rsync should never be hosted on ipv4 clearnet where a chinese bot will find it 3 secs after the route is established after boot.
kbolino 3y ago
Even though sshd must run as root (in the usual case), it doesn't need unfettered access to kernel memory, most of the filesystem, most other processes, etc. However, you could only really sandbox sshd-as-root. In order for sshd to do its job, it does need to be able to masquerade as arbitrary non-root users. That's still pretty bad but generally not "undetectably alter the operating system or firmware" bad.
- sweetjuly 3y ago
  >Even though sshd must run as root (in the usual case), it doesn't need unfettered access to kernel memory, most of the filesystem, most other processes, etc This is sort of overlooking the problem. While true, the processes spawned by sshd do need to be able to do all these things and so even if you did sandbox it, preserving functionality would all but guarantee an escape is trivial (...just spawn bash?).
  - kbolino 3y ago
    SELinux context is passed down to child processes. If sshd is running as confined root (system_u:system_r:sshd_t or similar), then the bash spawned by RCE will be too. Even if sshd is allowed to masquerade as an unconfined non-root user, that user will (regardless of SELinux) be unable to read or write /dev/kmem, ignore standard file permissions, etc.
    - sweetjuly 3y ago
      That's my point though--users expect to be able to do those things over ssh. Sandboxing sshd is hard because its child processes are expected to be able to do anything that an admin sitting at the console could do, up to and including reading/writing kernel memory.
      - kbolino 3y ago
        I'm assuming SSH root login is disabled and sudo requires separate authentication to elevate, but yeah, if there's a way to elevate yourself to unconfined root trivially after logging in, this doesn't buy you anything. Now, sandboxing sudo (in the general case) with SELinux probably isn't possible.
saltcured 3y ago
You could refactor sshd so most network payload processing is delegated to sandboxed sub-processes. Then an RCE there has less capabilities to exploit directly. But, I think you would have to assume an RCE can cause the sub-process to produce wrong answers. So if the answers are authorization decisions, you can transitively turn those wrong answers into RCE in the normal login or remote command execution context. But, the normal login or remote command execution is at least audited. And it might have other enforcement of which accounts or programs are permitted. A configuration disallowing root could not be bypassed by the sub-process. You could also decide to run all user logins/commands under some more confined SE-Linux process context. Then, the actual user sessions would be sandboxed compared to the real local root user. Of course, going too far with this may interfere with the desired use cases for SSH.
- asveikau 3y ago
  I thought that OpenSSH's sshd already separates itself into a privileged process and a low-privilege process. I don't know any details about that. Here's what Google showed me for that: https://github.com/openssh/openssh-portable/blob/master/README.privsep https://github.com/openssh/openssh-portable/blob/master/READ...
  - treasy 3y ago
    If you look at the diagram of privsep, the authentication process is part of the privileged binary, which is where this RCE lives http://www.citi.umich.edu/u/provos/ssh/priv.jpg http://www.citi.umich.edu/u/provos/ssh/priv.jpg
    - cryptonector 3y ago
      The signature validation could be moved into an unprivileged process forked from that one.
      - littlestymaar 3y ago
        That's an easy thing to say after the fact indeed but yes. In fact after such a disastrous backdoor I wouldn't be surprised if OpenSSH moved all code calling external libraries to unprivileged processes to make sure such an attack can never have such a dramatic effect (an auth bypass would still likely be possible, but that's still way better than a root RCE…). At this point “All libraries could be malicious” is a threat model that must be considered for something as security critical as OpenSSH.
treasy 3y ago
You can definitely prevent a lot of file/executable accesses via SELinux by running sshd in the default sshd_t or even customizing your own sshd domain and preventing sshd from being able to run binaries in its own domain without a transition. What you cannot prevent though is certain things that sshd _requires_ to function like certain capabilities and networking access. by default sshd has access to all files in /home/$user/.ssh/, but that could be prevented by giving private keys a new unique file context, etc. SELinux would not prevent all attacks, but it can mitigate quite a few as part of a larger security posture
- transpute 3y ago
  https://news.ycombinator.com/item?id=39879559 https://news.ycombinator.com/item?id=39879559 > Libselinux pulls in liblzma too
  - treasy 3y ago
    libselinux is the userspace tooling for selinux, it is irrelevant to this specific discussion as the backdoor does not target selinux in any way, and sshd does not have the capabilities required to make use of the libselinux tooling anyway libselinux is just an unwitting vector to link liblzma with openssh

ajross 3y ago

It's possible to spawn a sshd as an unprivileged or partially-capabilitized process. Such as sandbox isn't the default deployment, but it's done often enough and would work as designed to prevent privilege elevation above the sshd process.

admax88qqq 3y ago
How can sshd spawn interactive sessions for other users if it's sandboxed?
- ajross 3y ago
  Plausibly by having set-user-ID capability but not others an attacker might need. But in the more common case it just doesn't: you have an sshd running on a dedicated port for the sole purpose of running some service or another under a specific sandboxed UID. That's basically the github business model, for example.
  - dotancohen 3y ago
    I need full filesystem access, VIM, ls, cd, grep, awk, df, du at the very least. Sometimes perl, find, ncdu, and other utilities are necessary as well. Are you suggesting that each tool have its own SSH process wrapping it? Maybe write a shell to coordinate between them? It should support piping and output redirection, please.
    - ajross 3y ago
      Sigh. I'm not saying there's a sandboxed sshd setup that has equivalent functionality to the default one in your distro. I'm not even saying that there's one appropriate for your app. I'm saying, as a response to the point above, that sandboxing sshd is absolutely a valid defense-in-depth technique for privilege isolation, that it would work against attacks like this one to prevent whole-system exploitation, and that it's very commonly deployed in practice (c.f. running a git/ssh server a-la github).
      - semiquaver 3y ago
        Git’s use of the ssh protocol as a transport is a niche use case that ignores the actual problem. No one is seriously arguing that you can’t sandbox that constrained scenario but it’s not really relevant since it’s not the main purpose of the secure shell daemon.
      - dotancohen 3y ago
        The focus on the first S is good, yes, but SSH has another S and an H that needs focus as well.
  - deleted 3y ago
    [deleted]
- kbolino 3y ago
  SELinux does not rely on the usual UID/GID to determine what a process can do. System services, even when running as "root", are running as confined users in SELinux. Confined root cannot do anything which SELinux policy does not allow it to do. This means you can let sshd create new sessions for non-root users while still blocking it from doing the other things which unconfined root would be able to do. This is still a lot of power but it's not the godlike access which a person logged in as (unconfined) root has.
  - quotemstr 3y ago
    Doesn't matter. A malicious sshd able to run commands arbitrary users can just run malicious commands as those users. We'd need something more like a cryptographically attested setreuid() and execve() combination that would run only commands signed with the private key of the intended user. You'd want to use a shared clock or something to protect against replay attacks
    - kbolino 3y ago
      Yes, this won't directly protect against an attacker whose goal is to create a botnet, mine some crypto on your dime, etc. However, it will protect against corruption of the O/S itself and, in tandem with other controls, can limit the abilities an attacker has, and ensure things like auditing are still enforced (which can be tied to monitoring, and also used for forensics). Whether it's worth it or not depends on circumstances. In many cloud environments, nuking the VM instance and starting over is probably easier than fiddling with SELinux.
      - CanaryLayout 3y ago
        even easier is to STOP HOSTING SSHD ON IPV4 ON CLEARNET at minimum, ipv6 only if you absolutely must do it (it absolutely cuts the scans way down) better is to only host it on vpn even better is to only activate it with a portknocker, over vpn even better-better is to set up a private ipv6 peer-to-peer cloud and socat/relay to the private ipv6 network (yggdrasil comes to mind, but there's other solutions to darknet) your sshd you need for server maintenance/scp/git/rsync should never be hosted on ipv4 clearnet where a chinese bot will find it 3 secs after the route is established after boot.

kbolino 3y ago

Even though sshd must run as root (in the usual case), it doesn't need unfettered access to kernel memory, most of the filesystem, most other processes, etc. However, you could only really sandbox sshd-as-root. In order for sshd to do its job, it does need to be able to masquerade as arbitrary non-root users. That's still pretty bad but generally not "undetectably alter the operating system or firmware" bad.

sweetjuly 3y ago
>Even though sshd must run as root (in the usual case), it doesn't need unfettered access to kernel memory, most of the filesystem, most other processes, etc This is sort of overlooking the problem. While true, the processes spawned by sshd do need to be able to do all these things and so even if you did sandbox it, preserving functionality would all but guarantee an escape is trivial (...just spawn bash?).
- kbolino 3y ago
  SELinux context is passed down to child processes. If sshd is running as confined root (system_u:system_r:sshd_t or similar), then the bash spawned by RCE will be too. Even if sshd is allowed to masquerade as an unconfined non-root user, that user will (regardless of SELinux) be unable to read or write /dev/kmem, ignore standard file permissions, etc.
  - sweetjuly 3y ago
    That's my point though--users expect to be able to do those things over ssh. Sandboxing sshd is hard because its child processes are expected to be able to do anything that an admin sitting at the console could do, up to and including reading/writing kernel memory.
    - kbolino 3y ago
      I'm assuming SSH root login is disabled and sudo requires separate authentication to elevate, but yeah, if there's a way to elevate yourself to unconfined root trivially after logging in, this doesn't buy you anything. Now, sandboxing sudo (in the general case) with SELinux probably isn't possible.

saltcured 3y ago

You could refactor sshd so most network payload processing is delegated to sandboxed sub-processes. Then an RCE there has less capabilities to exploit directly. But, I think you would have to assume an RCE can cause the sub-process to produce wrong answers. So if the answers are authorization decisions, you can transitively turn those wrong answers into RCE in the normal login or remote command execution context. But, the normal login or remote command execution is at least audited. And it might have other enforcement of which accounts or programs are permitted. A configuration disallowing root could not be bypassed by the sub-process. You could also decide to run all user logins/commands under some more confined SE-Linux process context. Then, the actual user sessions would be sandboxed compared to the real local root user. Of course, going too far with this may interfere with the desired use cases for SSH.

asveikau 3y ago
I thought that OpenSSH's sshd already separates itself into a privileged process and a low-privilege process. I don't know any details about that. Here's what Google showed me for that: https://github.com/openssh/openssh-portable/blob/master/README.privsep https://github.com/openssh/openssh-portable/blob/master/READ...
- treasy 3y ago
  If you look at the diagram of privsep, the authentication process is part of the privileged binary, which is where this RCE lives http://www.citi.umich.edu/u/provos/ssh/priv.jpg http://www.citi.umich.edu/u/provos/ssh/priv.jpg
  - cryptonector 3y ago
    The signature validation could be moved into an unprivileged process forked from that one.
    - littlestymaar 3y ago
      That's an easy thing to say after the fact indeed but yes. In fact after such a disastrous backdoor I wouldn't be surprised if OpenSSH moved all code calling external libraries to unprivileged processes to make sure such an attack can never have such a dramatic effect (an auth bypass would still likely be possible, but that's still way better than a root RCE…). At this point “All libraries could be malicious” is a threat model that must be considered for something as security critical as OpenSSH.

treasy 3y ago

You can definitely prevent a lot of file/executable accesses via SELinux by running sshd in the default sshd_t or even customizing your own sshd domain and preventing sshd from being able to run binaries in its own domain without a transition. What you cannot prevent though is certain things that sshd _requires_ to function like certain capabilities and networking access. by default sshd has access to all files in /home/$user/.ssh/, but that could be prevented by giving private keys a new unique file context, etc. SELinux would not prevent all attacks, but it can mitigate quite a few as part of a larger security posture

transpute 3y ago
https://news.ycombinator.com/item?id=39879559 https://news.ycombinator.com/item?id=39879559 > Libselinux pulls in liblzma too
- treasy 3y ago
  libselinux is the userspace tooling for selinux, it is irrelevant to this specific discussion as the backdoor does not target selinux in any way, and sshd does not have the capabilities required to make use of the libselinux tooling anyway libselinux is just an unwitting vector to link liblzma with openssh