7 ms·

Bubblewrap: Unprivileged sandboxing tool for Linux

189 points by varbhat 5y ago

akvadrako 5y ago
This seems to be the best of breed and it basically the same thing used by flatpak.
- makeworld 5y ago
  My understanding is that this is exactly what flatpak uses.
  - yjftsjthsd-h 5y ago
    My reading of https://github.com/containers/bubblewrap#related-project-comparison-firejail https://github.com/containers/bubblewrap#related-project-com... is that it is literally what flatpak uses: > Firejail is similar to Flatpak before bubblewrap was split out
LinuxBender 5y ago
This looks like a very useful tool especially for isolating old unsupported daemons and not having to write really complex selinux rules especially in places where others are tempted to just set selinux permissive. And not just because I like bubblewrap IRL. I see there are a few usage examples [1] out there. Are there any bigger collections of examples that people have run across? [1] - https://wiki.archlinux.org/title/Bubblewrap https://wiki.archlinux.org/title/Bubblewrap
skywal_l 5y ago
"How does it compare to firejail?" you were going to ask: https://github.com/containers/bubblewrap#related-project-comparison-firejail https://github.com/containers/bubblewrap#related-project-com.... You're welcome.
lvass 5y ago
>Also, @cgwalters thinks trying to whitelist file paths is a bad idea given the myriad ways users have to manipulate paths, and the myriad ways in which system administrators may configure a system. So you either get the entire filesystem or no file access? Isn't this a huge dealbreaker for almost everything?
- iancarroll 5y ago
  I think this is just a comment on Firejail's implementation, as I think Firejail runs as root and thus must enforce its own permission checks. It seems like bubblewrap uses a mount namespace created by the current user which would allow controlling access without any special checks.
  - dj_gitmo 5y ago
    I'm running firejail now and It doesn't look like it runs as root.
    - anderspitman 5y ago
      Correct, it is an SUID executable[0], as is bubblewrap. [0]: https://en.m.wikipedia.org/wiki/Setuid https://en.m.wikipedia.org/wiki/Setuid
      - jwilk 5y ago
        This is poorly documented, but if the kernel supports unprivileged user namespaces, bubblewrap works without suid.
        anderspitman 5y ago
        Good to know, thanks.
        yjftsjthsd-h 5y ago
        Does/can firejail, or is that a difference?
jamesdutc 5y ago
Bubblewrap is a surprisingly useful tool for general system administration tasks. Even though the documentation claims that "[y]ou are unlikely to use it directly from the commandline, although that is possible," I use it as a helper tool in this mode very frequently. This can be very useful for debugging since, for example, you can `bwrap --ro-bind / / --tmpfs ~ $SHELL` to get a "clean" shell in which you can isolate yourself from the effect of configuration dotfiles and can even `--ro-bind my-hosts /etc/hosts` to simulate certain system-level state (without requiring a full VM, heavier container, or root access.) Of course, I've also written some simple shell scripts around `bwrap` to make this all a bit simpler (since this quickly reaches `qemu`-levels of argv proliferation.)
- kzrdude 5y ago
  Nice! The other day I wanted to do something like this - ended up with docker to have a clean HOME, but bubblewrap might be easier and faster.
lapinot 5y ago
Does anyone know if it's possible to configure bubblewrap to attach to an existing namespace (network namespace in my case)? I'm selfhosting stuff and wanted a diy container runtime but i can't seem to isolate stuff from the system and still retain some linking between some parts (ie doing stuff like what a docker-compose file specifies).
- fock 5y ago
  I enter bwrap namespaces with classic nsenter (bwrap has no option), doesn't this work for you?
  - lapinot 5y ago
    Thanks, that should do it indeed, i'll try that in my scripts! Tbh i didn't search long, i was just a bit disappointed that bwrap didn't have this capability. (because now i have the choice of doing nsenter outside or inside of bwrap.. on the top of my head that shouldn't make a difference if the kinds of namespaces touched is disjoint, but i'm sure there are some subtle bugs waiting to be introduced)
apitman 5y ago
Still requires SUID. It makes no sense to have to increase privileges in order to be able to decrease privileges. Any process should be able to spawn a child process with strictly lesser privileges, including granular cpu, memory, network, and filesystem (including path and size restrictions) access. It's sad there isn't a simple, standard cross-platform API for doing this. I would be fine having to rewrite my apps to take advantage of some fancy capabilities-based security paradigm, but give me realistic APIs to do so. I'm currently experimenting with shipping apps as QEMU VMs, packaging QEMU and a minimal kernel with the app. It works surprisingly well, even on Windows with full x86 emulation. And with their newish WHPX API (basically kvm for Windows) the future might be really exciting. EDIT: Someone on the Firejail thread says bubblewrap can be compiled non-SUID and that's the common usage these days. I need to look into this more.
- mwcampbell 5y ago
  I've also been looking into shipping apps as VM images with a minimal kernel. Do you know if WHPX requires the user to have admin rights? On the host side, Windows and Mac ports of crosvm [1] could be useful. crosvm seems to have all the necessary virtio device types, but a greater focus on security than QEMU. [1]: https://google.github.io/crosvm/ https://google.github.io/crosvm/
  - anderspitman 5y ago
    WHPX does not require admin in my testing, but it does require Windows Pro and several virtualization features to be enabled. It's not too bad but still not ideal. Currently I'm not banking on it though and focusing my testing assuming full emulation on older hardware. If you're interested in collaborating on this feel free to email me or open a thread over on https://forum.indiebits.io/ https://forum.indiebits.io/ EDIT: Also, Firecracker seems to have more steam behind it than crosvm. I really wish it ran on Windows. That said, QEMU does have a microvm mode[0] that looks very interesting. [0]: https://mergeboard.com/blog/2-qemu-microvm-docker/ https://mergeboard.com/blog/2-qemu-microvm-docker/
    - mwcampbell 5y ago
dang 5y ago
Related: Bubblewrap: Unprivileged sandboxing tool - https://news.ycombinator.com/item?id=12241971 https://news.ycombinator.com/item?id=12241971 - Aug 2016 (8 comments)
mstef 5y ago
some time ago i made a comparison between different jailing tools: https://ctrlc.hu/~stef/jails.txt https://ctrlc.hu/~stef/jails.txt
- remram 5y ago
  Readable, accessible, responsive version: https://gist.github.com/remram44/fe30af580bfe1494931a392e95711207 https://gist.github.com/remram44/fe30af580bfe1494931a392e957...
- ranguna 5y ago
  I know HN people are all about simplicity, but just wanted to let you know that your txt file is unreadable on mobile.
rsa25519 5y ago
Note that a sandbox escape is often possible via TIOCSTI (CVE-2017-5226) [0] unless a special flag (--new-session) is used. Bubblewrap is aware of this, yet their documentation gives no indication that this flag is necessary to produce a secure sandbox. In --help, the documentation of --new-session is simply "Create a new terminal session," which severely understates its importance. It's frustrating to have such a useful tool be knowingly easy to misuse. [0]: https://github.com/containers/bubblewrap/issues/142 https://github.com/containers/bubblewrap/issues/142
- jagrsw 5y ago
  FWIW, it's disabled in nsjail by default https://github.com/google/nsjail/blob/6483728e2490c1fc497a81bba5682515eb489cf8/config.proto#L130 https://github.com/google/nsjail/blob/6483728e2490c1fc497a81... with relevant comment.
- ciupicri 5y ago
  By the way, it looks like OpenBSD disabled TIOCSTI [1] [2]. [1]: https://undeadly.org/cgi?action=article;sid=20170701132619 https://undeadly.org/cgi?action=article;sid=20170701132619 [2]: https://marc.info/?l=openbsd-cvs&m=149870941319610 https://marc.info/?l=openbsd-cvs&m=149870941319610
rosetremiere 5y ago
Here is something I'd like to have in fedora (or desktop linux in general): A sort of “workspace manager” using bubblewrap for isolating user-defined groups of applications and file hierarchy. I imagine it as some kind of GUI with “boxes”, each defining a “workspace” with specified permissions (access to that and that folder, network, etc), such that the user could easily drag and drop apps into new workspaces. This way, I could have a “banking” workspace just containing, say, firefox, and a “work” workspace with, say, firefox and thunderbird and whatever, etc. The “workspaces” would by default be as unprivileged as possible, with the possibility to give access to folders, or even maybe to use unionfs or similar to combine the views of different workspaces. For ease of use, I could assign a color or icon to each workspace, and the manager would automatically generate desktop files so that I would easily be able to spot “banking firefox” from “work firefox”. I feel like with bubblewrap, this is only a short python GUI away, and would be super helpful to strengthen security on linux, even more so for non-technically inclined, or lazy people. [Edit:] If someone wants to discuss things further and even maybe try and write a “MVP”, I'd be happy to try.
- seanhunter 5y ago
  This is almost exactly the concept behind qubes-os except it uses the xen hypervisor instead of bubblewrap and vms for boxes. https://www.qubes-os.org/ https://www.qubes-os.org/
  - rosetremiere 5y ago
    Right. I'm being a bit less ambitious here: I've recommended fedora to plenty of (non-tech) people around me, and fedora presumably already has all the necessary tools to add such a simple “workspace manager” without much disruption. It would provide an easy intuitive step-up in terms of security, starting from something people already know.
- fock 5y ago
  First problem is that proper (incl. X11...) isolation costs you 3D (so not great for Firefox). Otherwise there is "toolbox" by Fedora, which you could surely combine with Xephyr for a wine-like, more isolated experience. I do that with a shell-only workflow (though on my HiDPI-machines I still have to patch Xephyr to scale the output (which is pretty easy using its GL-backend tbh, but I need to refine my amateur C dabblings to release that). EDIT: I'm happy to discuss how to use Wayland with proper isolation and 3D! (and some frame-decoration)
usr1106 5y ago
Who is https://github.com/containers https://github.com/containers? Well, I see the member list and recognize some names as podman developers. But is there any presentation, policy, statutes...?
- WhyNotHugo 5y ago
  AFAIK it's Red Hat.
mobiclick 5y ago
I really wish bubblewrap had a way to mount filesystems as copy-on-write instead of read-only. Having ways to impose limits on memory would be nice too.

189 points by varbhat 5y ago

akvadrako 5y ago

This seems to be the best of breed and it basically the same thing used by flatpak.

makeworld 5y ago
My understanding is that this is exactly what flatpak uses.
- yjftsjthsd-h 5y ago
  My reading of https://github.com/containers/bubblewrap#related-project-comparison-firejail https://github.com/containers/bubblewrap#related-project-com... is that it is literally what flatpak uses: > Firejail is similar to Flatpak before bubblewrap was split out

LinuxBender 5y ago

This looks like a very useful tool especially for isolating old unsupported daemons and not having to write really complex selinux rules especially in places where others are tempted to just set selinux permissive. And not just because I like bubblewrap IRL. I see there are a few usage examples [1] out there. Are there any bigger collections of examples that people have run across? [1] - https://wiki.archlinux.org/title/Bubblewrap https://wiki.archlinux.org/title/Bubblewrap

skywal_l 5y ago

"How does it compare to firejail?" you were going to ask: https://github.com/containers/bubblewrap#related-project-comparison-firejail https://github.com/containers/bubblewrap#related-project-com.... You're welcome.

lvass 5y ago

>Also, @cgwalters thinks trying to whitelist file paths is a bad idea given the myriad ways users have to manipulate paths, and the myriad ways in which system administrators may configure a system. So you either get the entire filesystem or no file access? Isn't this a huge dealbreaker for almost everything?

iancarroll 5y ago
I think this is just a comment on Firejail's implementation, as I think Firejail runs as root and thus must enforce its own permission checks. It seems like bubblewrap uses a mount namespace created by the current user which would allow controlling access without any special checks.
- dj_gitmo 5y ago
  I'm running firejail now and It doesn't look like it runs as root.
  - anderspitman 5y ago
    Correct, it is an SUID executable[0], as is bubblewrap. [0]: https://en.m.wikipedia.org/wiki/Setuid https://en.m.wikipedia.org/wiki/Setuid
    - jwilk 5y ago
      This is poorly documented, but if the kernel supports unprivileged user namespaces, bubblewrap works without suid.
      - anderspitman 5y ago
        Good to know, thanks.
      - yjftsjthsd-h 5y ago
        Does/can firejail, or is that a difference?

jamesdutc 5y ago

Bubblewrap is a surprisingly useful tool for general system administration tasks. Even though the documentation claims that "[y]ou are unlikely to use it directly from the commandline, although that is possible," I use it as a helper tool in this mode very frequently. This can be very useful for debugging since, for example, you can `bwrap --ro-bind / / --tmpfs ~ $SHELL` to get a "clean" shell in which you can isolate yourself from the effect of configuration dotfiles and can even `--ro-bind my-hosts /etc/hosts` to simulate certain system-level state (without requiring a full VM, heavier container, or root access.) Of course, I've also written some simple shell scripts around `bwrap` to make this all a bit simpler (since this quickly reaches `qemu`-levels of argv proliferation.)

kzrdude 5y ago
Nice! The other day I wanted to do something like this - ended up with docker to have a clean HOME, but bubblewrap might be easier and faster.

lapinot 5y ago

Does anyone know if it's possible to configure bubblewrap to attach to an existing namespace (network namespace in my case)? I'm selfhosting stuff and wanted a diy container runtime but i can't seem to isolate stuff from the system and still retain some linking between some parts (ie doing stuff like what a docker-compose file specifies).

fock 5y ago
I enter bwrap namespaces with classic nsenter (bwrap has no option), doesn't this work for you?
- lapinot 5y ago
  Thanks, that should do it indeed, i'll try that in my scripts! Tbh i didn't search long, i was just a bit disappointed that bwrap didn't have this capability. (because now i have the choice of doing nsenter outside or inside of bwrap.. on the top of my head that shouldn't make a difference if the kinds of namespaces touched is disjoint, but i'm sure there are some subtle bugs waiting to be introduced)

apitman 5y ago

Still requires SUID. It makes no sense to have to increase privileges in order to be able to decrease privileges. Any process should be able to spawn a child process with strictly lesser privileges, including granular cpu, memory, network, and filesystem (including path and size restrictions) access. It's sad there isn't a simple, standard cross-platform API for doing this. I would be fine having to rewrite my apps to take advantage of some fancy capabilities-based security paradigm, but give me realistic APIs to do so. I'm currently experimenting with shipping apps as QEMU VMs, packaging QEMU and a minimal kernel with the app. It works surprisingly well, even on Windows with full x86 emulation. And with their newish WHPX API (basically kvm for Windows) the future might be really exciting. EDIT: Someone on the Firejail thread says bubblewrap can be compiled non-SUID and that's the common usage these days. I need to look into this more.

mwcampbell 5y ago
I've also been looking into shipping apps as VM images with a minimal kernel. Do you know if WHPX requires the user to have admin rights? On the host side, Windows and Mac ports of crosvm [1] could be useful. crosvm seems to have all the necessary virtio device types, but a greater focus on security than QEMU. [1]: https://google.github.io/crosvm/ https://google.github.io/crosvm/
- anderspitman 5y ago
  WHPX does not require admin in my testing, but it does require Windows Pro and several virtualization features to be enabled. It's not too bad but still not ideal. Currently I'm not banking on it though and focusing my testing assuming full emulation on older hardware. If you're interested in collaborating on this feel free to email me or open a thread over on https://forum.indiebits.io/ https://forum.indiebits.io/ EDIT: Also, Firecracker seems to have more steam behind it than crosvm. I really wish it ran on Windows. That said, QEMU does have a microvm mode[0] that looks very interesting. [0]: https://mergeboard.com/blog/2-qemu-microvm-docker/ https://mergeboard.com/blog/2-qemu-microvm-docker/
  - mwcampbell 5y ago

dang 5y ago

Related: Bubblewrap: Unprivileged sandboxing tool - https://news.ycombinator.com/item?id=12241971 https://news.ycombinator.com/item?id=12241971 - Aug 2016 (8 comments)

mstef 5y ago

some time ago i made a comparison between different jailing tools: https://ctrlc.hu/~stef/jails.txt https://ctrlc.hu/~stef/jails.txt

remram 5y ago
Readable, accessible, responsive version: https://gist.github.com/remram44/fe30af580bfe1494931a392e95711207 https://gist.github.com/remram44/fe30af580bfe1494931a392e957...
ranguna 5y ago
I know HN people are all about simplicity, but just wanted to let you know that your txt file is unreadable on mobile.

rsa25519 5y ago

Note that a sandbox escape is often possible via TIOCSTI (CVE-2017-5226) [0] unless a special flag (--new-session) is used. Bubblewrap is aware of this, yet their documentation gives no indication that this flag is necessary to produce a secure sandbox. In --help, the documentation of --new-session is simply "Create a new terminal session," which severely understates its importance. It's frustrating to have such a useful tool be knowingly easy to misuse. [0]: https://github.com/containers/bubblewrap/issues/142 https://github.com/containers/bubblewrap/issues/142

jagrsw 5y ago
FWIW, it's disabled in nsjail by default https://github.com/google/nsjail/blob/6483728e2490c1fc497a81bba5682515eb489cf8/config.proto#L130 https://github.com/google/nsjail/blob/6483728e2490c1fc497a81... with relevant comment.
ciupicri 5y ago
By the way, it looks like OpenBSD disabled TIOCSTI [1] [2]. [1]: https://undeadly.org/cgi?action=article;sid=20170701132619 https://undeadly.org/cgi?action=article;sid=20170701132619 [2]: https://marc.info/?l=openbsd-cvs&m=149870941319610 https://marc.info/?l=openbsd-cvs&m=149870941319610

rosetremiere 5y ago

Here is something I'd like to have in fedora (or desktop linux in general): A sort of “workspace manager” using bubblewrap for isolating user-defined groups of applications and file hierarchy. I imagine it as some kind of GUI with “boxes”, each defining a “workspace” with specified permissions (access to that and that folder, network, etc), such that the user could easily drag and drop apps into new workspaces. This way, I could have a “banking” workspace just containing, say, firefox, and a “work” workspace with, say, firefox and thunderbird and whatever, etc. The “workspaces” would by default be as unprivileged as possible, with the possibility to give access to folders, or even maybe to use unionfs or similar to combine the views of different workspaces. For ease of use, I could assign a color or icon to each workspace, and the manager would automatically generate desktop files so that I would easily be able to spot “banking firefox” from “work firefox”. I feel like with bubblewrap, this is only a short python GUI away, and would be super helpful to strengthen security on linux, even more so for non-technically inclined, or lazy people. [Edit:] If someone wants to discuss things further and even maybe try and write a “MVP”, I'd be happy to try.

seanhunter 5y ago
This is almost exactly the concept behind qubes-os except it uses the xen hypervisor instead of bubblewrap and vms for boxes. https://www.qubes-os.org/ https://www.qubes-os.org/
- rosetremiere 5y ago
  Right. I'm being a bit less ambitious here: I've recommended fedora to plenty of (non-tech) people around me, and fedora presumably already has all the necessary tools to add such a simple “workspace manager” without much disruption. It would provide an easy intuitive step-up in terms of security, starting from something people already know.
fock 5y ago
First problem is that proper (incl. X11...) isolation costs you 3D (so not great for Firefox). Otherwise there is "toolbox" by Fedora, which you could surely combine with Xephyr for a wine-like, more isolated experience. I do that with a shell-only workflow (though on my HiDPI-machines I still have to patch Xephyr to scale the output (which is pretty easy using its GL-backend tbh, but I need to refine my amateur C dabblings to release that). EDIT: I'm happy to discuss how to use Wayland with proper isolation and 3D! (and some frame-decoration)

usr1106 5y ago

Who is https://github.com/containers https://github.com/containers? Well, I see the member list and recognize some names as podman developers. But is there any presentation, policy, statutes...?

WhyNotHugo 5y ago
AFAIK it's Red Hat.

mobiclick 5y ago

I really wish bubblewrap had a way to mount filesystems as copy-on-write instead of read-only. Having ways to impose limits on memory would be nice too.