3 ms·

Wouldn't in-place decompression for a "self-executable" squashfs require one to disable NX for that binary? That seems like it would be a terrible idea from a s

Wouldn't in-place decompression for a "self-executable" squashfs require one to disable NX for that binary? That seems like it would be a terrible idea from a security perspective. (Even for pure data, in-place decompression seems like it would heavily interfere with e.g. demand paging of such data into RAM, using the binary itself as backing store. Disk space is so cheap these days - I don't think things like squashfs should be used nowadays unless truly necessary.)

dullgiulio 8y ago
Yes, but for the same reason /tmp is also often a non-executable partition.
_wmd 8y ago
Physical pages can be mapped more than once, it is the mapping that describes protection bits. Implementing an in-place FUSE filesystem for example would have no bearing on whether other mappings of the same file lacked execute bits, and in the case of the FUSE interface, the kernel does not even know the original source of the data I think this could also be achieved with a loopback device setup to cover some of the file, again no relevance to NX etc, but that would require setuid or similar to configure the device
- pritambaral 8y ago
  With the recent (with upcoming improvements) userspace mount support in Linux, even suid may be unnecessary.
pritambaral 8y ago
It doesn't have to be in-place, or even compressed; it just has to be automatic and OS-integrated. With something like squashfs (with or without compression) the kernel handles all the paging, mapping, and decompression (if any). Of course there are the classical benefits like resource usage being more efficient, but more importantly, less userspace code that needs to worry/care about filesystems and file layout etc.
- zozbot123 8y ago
  > It doesn't have to be in-place, or even compressed; it just has to be automatic and OS-integrated. With something like squashfs (with or without compression) the kernel handles all the paging, mapping, and decompression (if any). Sure, but I don't think you can do that within a self-contained binary, the kernel doesn't really support that. It would have to just be a loopback image, and lose the whole "self-contained binary" angle that makes this different from all sorts of other container tech.
  - pritambaral 8y ago
    > Sure, but I don't think you can do that within a self-contained binary, the kernel doesn't really support that. An executable binary can set up a mount namespace, mount /proc/self/exe or argv[0] (with some offset) as a loopback device within the namespace, and then execute a process within the mount.
    - hedora 8y ago
      It would probably only be a few lines of portable shell code. Mounting at an offset is one line; runing the executable is a second. Not sure what you have to do to manually drop into a container. (Though, I think you need to be root to run those lines, and you could put setuid root binaries in the image...)
      - pritambaral 8y ago
        > Though, I think you need to be root to run those lines If you're thinking of mounts, Linux is getting there: https://lwn.net/Articles/697278/ https://lwn.net/Articles/697278/ and https://lwn.net/Articles/755498/ https://lwn.net/Articles/755498/ If you're thinking containerisation, it has been possible for a while, and new improvements are coming up: https://www.youtube.com/watch?v=-PZNF8XDNn8 https://www.youtube.com/watch?v=-PZNF8XDNn8 and https://events.linuxfoundation.org/wp-content/uploads/2017/12/2018-LSS-Europe-Edinburgh-Namespaces-and-Capabilities_Christian-Brauner.pdf https://events.linuxfoundation.org/wp-content/uploads/2017/1...
exrook 8y ago
NX does not need to be disabled[1] for the binary thanks to W^X[0]. As long as pages are never mapped as writeable and executable at the same time, you don't lose any protection. [0] https://en.wikipedia.org/wiki/W%5EX https://en.wikipedia.org/wiki/W%5EX [1] AFAIK, on Linux NX isn't something that is enabled or disabled for individual programs anyways, it's up to the userland whether or not it takes advantage of the mmap(2) mapping flags
- bdonlan 8y ago
  In general, you lose some protection, as a two-stage exploit might write some data in W state then flip to (or wait for) X state for execution. That being said, a self-decompressing binary would only do this at startup, before it consumes untrusted input, so given a way to drop map-executable permissions that wouldn't be a problem.
mwcampbell 8y ago
> Disk space is so cheap these days And yet we always find new ways to waste it. 16 GB was once an unimaginable amount of space, and yet one of my coworkers now takes it for granted that 16 GB isn't enough storage for a smartphone. We should do our best not to waste any more.
- mroche 8y ago
  Agreed. We had a petabyte storage for scratch data on our cluster. We were stuck around 95-100% capacity used for the past month and some change until we recently added another half petabyte or so. We were constantly deleting and automating warnings to people that info would be removed and their access restricted if things weren’t removed. We ended up suspending the cluster for a day while we tried to figure out how we were exponentially losing data (one user decided to start uploading terabytes of data to scratch, which is against policy). No one seemed to understand that if data filled up completely, the entire cluster would go offline because _we wouldn’t have the space for anything_.
  - tfigment 8y ago
    Cynically, I think its not that they don't understand its that they don't care and that it is why you are probably there to care for them and solve it. I have similar engineers and its a struggle. Didn't Douglas Adams invent the concept of the SEP Field to explain this (somebody else's problem)?
    - gnufx 8y ago
      Yes, it's part of system management to prevent users interfering with one another -- and at least detect problems that are difficult to prevent, like activity crippling a parallel filesystem, and sorting it out.

dullgiulio 8y ago

Yes, but for the same reason /tmp is also often a non-executable partition.

_wmd 8y ago

Physical pages can be mapped more than once, it is the mapping that describes protection bits. Implementing an in-place FUSE filesystem for example would have no bearing on whether other mappings of the same file lacked execute bits, and in the case of the FUSE interface, the kernel does not even know the original source of the data I think this could also be achieved with a loopback device setup to cover some of the file, again no relevance to NX etc, but that would require setuid or similar to configure the device

pritambaral 8y ago
With the recent (with upcoming improvements) userspace mount support in Linux, even suid may be unnecessary.

pritambaral 8y ago

It doesn't have to be in-place, or even compressed; it just has to be automatic and OS-integrated. With something like squashfs (with or without compression) the kernel handles all the paging, mapping, and decompression (if any). Of course there are the classical benefits like resource usage being more efficient, but more importantly, less userspace code that needs to worry/care about filesystems and file layout etc.

zozbot123 8y ago
> It doesn't have to be in-place, or even compressed; it just has to be automatic and OS-integrated. With something like squashfs (with or without compression) the kernel handles all the paging, mapping, and decompression (if any). Sure, but I don't think you can do that within a self-contained binary, the kernel doesn't really support that. It would have to just be a loopback image, and lose the whole "self-contained binary" angle that makes this different from all sorts of other container tech.
- pritambaral 8y ago
  > Sure, but I don't think you can do that within a self-contained binary, the kernel doesn't really support that. An executable binary can set up a mount namespace, mount /proc/self/exe or argv[0] (with some offset) as a loopback device within the namespace, and then execute a process within the mount.
  - hedora 8y ago
    It would probably only be a few lines of portable shell code. Mounting at an offset is one line; runing the executable is a second. Not sure what you have to do to manually drop into a container. (Though, I think you need to be root to run those lines, and you could put setuid root binaries in the image...)
    - pritambaral 8y ago
      > Though, I think you need to be root to run those lines If you're thinking of mounts, Linux is getting there: https://lwn.net/Articles/697278/ https://lwn.net/Articles/697278/ and https://lwn.net/Articles/755498/ https://lwn.net/Articles/755498/ If you're thinking containerisation, it has been possible for a while, and new improvements are coming up: https://www.youtube.com/watch?v=-PZNF8XDNn8 https://www.youtube.com/watch?v=-PZNF8XDNn8 and https://events.linuxfoundation.org/wp-content/uploads/2017/12/2018-LSS-Europe-Edinburgh-Namespaces-and-Capabilities_Christian-Brauner.pdf https://events.linuxfoundation.org/wp-content/uploads/2017/1...

exrook 8y ago

NX does not need to be disabled[1] for the binary thanks to W^X[0]. As long as pages are never mapped as writeable and executable at the same time, you don't lose any protection. [0] https://en.wikipedia.org/wiki/W%5EX https://en.wikipedia.org/wiki/W%5EX [1] AFAIK, on Linux NX isn't something that is enabled or disabled for individual programs anyways, it's up to the userland whether or not it takes advantage of the mmap(2) mapping flags

bdonlan 8y ago
In general, you lose some protection, as a two-stage exploit might write some data in W state then flip to (or wait for) X state for execution. That being said, a self-decompressing binary would only do this at startup, before it consumes untrusted input, so given a way to drop map-executable permissions that wouldn't be a problem.

mwcampbell 8y ago

> Disk space is so cheap these days And yet we always find new ways to waste it. 16 GB was once an unimaginable amount of space, and yet one of my coworkers now takes it for granted that 16 GB isn't enough storage for a smartphone. We should do our best not to waste any more.

mroche 8y ago
Agreed. We had a petabyte storage for scratch data on our cluster. We were stuck around 95-100% capacity used for the past month and some change until we recently added another half petabyte or so. We were constantly deleting and automating warnings to people that info would be removed and their access restricted if things weren’t removed. We ended up suspending the cluster for a day while we tried to figure out how we were exponentially losing data (one user decided to start uploading terabytes of data to scratch, which is against policy). No one seemed to understand that if data filled up completely, the entire cluster would go offline because _we wouldn’t have the space for anything_.
- tfigment 8y ago
  Cynically, I think its not that they don't understand its that they don't care and that it is why you are probably there to care for them and solve it. I have similar engineers and its a struggle. Didn't Douglas Adams invent the concept of the SEP Field to explain this (somebody else's problem)?
  - gnufx 8y ago
    Yes, it's part of system management to prevent users interfering with one another -- and at least detect problems that are difficult to prevent, like activity crippling a parallel filesystem, and sorting it out.