6 ms·

Using bubblewrap to add sandboxing to NetBSD

103 points by jaypatelani 11mo ago

jmclnx 11mo ago
Always nice to see NetBSD posts here, that nice OS gets hardly any press.
lovegrenoble 11mo ago
bubblewrap popping )) https://brainteaser.top/bubblespop.html https://brainteaser.top/bubblespop.html
aborsy 11mo ago
I experimented bubblewrap as a better alternative to firejail. Unfortunately there are scripts for few applications. It’s tool for developers. Users are better off flatpaks powered by bubblewrap .
- johnisgood 11mo ago
  I have been using firejail for a long time now. It seem to work well for me. At times I have to write my own profiles, yeah.
  - aborsy 11mo ago
    The situation with firejail is different. There are a lot of profiles, and they are easier to customize and write. But firejail needs root privilege, so there are arguments that it may actually reduce the security, because even though it sandboxes some applications, non-privilege processes may gain privilege if there is a flaw in the fire jail itself. Firejail is also criticized for complexity.
    - johnisgood 11mo ago
      Firejail does not need root privileges at all. You are probably referring to SUID. It is a "setuid ELF 64-bit" executable. I run it with my own user only. Complexity? It just supports a lot of stuff. See the manual page. It seems simple enough to me.
      - aborsy 11mo ago
        Yeah, I meant the same thing! If you run it under your non-root user, and there is a vulnerability in firejail, the jailed process could run as root, right? Complexity referred to large code base and attack surface. I haven’t tried it myself. I would be interested knowing, whether it might lower the security in some cases. Maybe it’s useful to sandbox apps that don’t have flatpaks.
        jonway 11mo ago
        Yeah, it runs with escalated privs. People wager that SUID binaries are extremely risky and this seems totally reasonable to me! I don't know where to find a really good security analysis of the risks, however. Firejail is most likely not bullet-proof, although not soft by any stretch of the imagination and I don't know how often and how many security researchers are profiling this tool. My gut sense is that flatpak gets much more scrutiny since it ships, and firejail is typically not shipped, but another package as far as I can tell (maybe in some specialized distros?).
        neobrain 11mo ago
        SUID by itself is not a security liability. Typically, programs using SUID quickly drop their additional privileges during startup to avoid the scenario you're describing. So the vulnerability in firejail would have to be early during startup before any of that. Still, it can happen of course, particularly with memory-unsafe languages. Conversely without root privileges you have to sacrifice strength of your sandbox, widening the vulnerability window in the app you were trying to constrain to begin with. Meanwhile bubblewrap relies on user namespaces, which come with their own set of security problems. They are disabled by default in many distributions for that reason.
- Imustaskforhelp 11mo ago
  I wanted to build flatpaks cli applications and I still don't understand how to build a flatpak cli in the first place and I have asked in some communities and they mention things like vim being in flatpak but I more so need a definitive guide on how to publish a cli application to flatpak A lot of focus in flatpak sadly gets focused on guis and not clis Like how much efforts would it take to convert a golang static binary/project cli to flatpak?
qwertox 11mo ago
Bubblewrap is a really interesting project, really worth checking out. https://github.com/containers/bubblewrap https://github.com/containers/bubblewrap It's the base for Flatpack, the thing that makes Flatpack be sandboxed. I use it to run Claude Code / Codex / Gemini CLI, to make sure that they have a limited / fake view of my system. You can bind directories into it, or overlay them into it, restrict other kinds of access. If Docker is a thing in a scale between a VM and your OS, Bubblewrap is a thing a scale between Docker and your OS. You use your OS, instead of installing and managing an OS like you do with Docker, but you get filesystem and process isolation like with Docker. Though I had an issue where I cannot use `--new-session`, which is kind of dangerous to not use, but you can get around it if you use seccomp to block ioctl calls, and ptrace.
- udev4096 11mo ago
  bubblewrap escapes are not unheard of. Infact, it's a common theme that the general linux landscape lacks strong sandboxing, even if you use bwrap, firejail, etc. Especially linux desktop, a security firehazard to say the least unless you are using QubesOS
- globular-toast 11mo ago
  I documented what I came up with for Claude Code: https://blog.gpkb.org/posts/ai-agent-sandbox/ https://blog.gpkb.org/posts/ai-agent-sandbox/ However, I couldn't get this to work for Codex, it kept failing at the auth bit and I couldn't figure out how to fix it. Anyone got a working solution for Codex?
- aborsy 11mo ago
  A question. How do you know what permissions are required by an application, to write a bubblewrap script? In AppArmor, you exercise the application and aa-logprof suggests permissions requested by application. If you know AppArmor, usually you can refine those suggestions and write a profile. It may not be ideal, as aa-logprof’s permissions are multiple choice suggestions, require user knowledge and may be too broad or specific, but it could work. You will see that there are many and all kinds of permissions, and there is no way that you will be able to guess them without aa-logprof. What is the equivalent of aa-logprof in bubblewrap and how do you find the required permissions?
DeathArrow 11mo ago
I would have loved to see something like OCI containers.
- yjftsjthsd-h 11mo ago
  This would underpin that, right? First you build the namespace primitives, then you put the handy wrapper on top that composes a root fs and spawns a container from it using those namespace primitives
matesz 11mo ago
As a side note NetBSD is THE ONLY operating system of which binaries are fully bit by bit reproducible. Moreover it vendores patched gcc compiler, so the entire toolchain is reproducible too as well. How cool is that?! For the record golang brings it further - its entire package registry containing +40mln packages is bit by bit reproducible.
- ksec 11mo ago
  I thought FreeBSD has it as well with v15?
  - Imustaskforhelp 11mo ago
    I think I have read something like that similar as well. Freebsd seems like an amazing project too. I am always confused tho by what is the actual difference b/w freebsd,netbsd,openbsd as I feel like freebsd is taking a lot of positive ideas from both netbsd and openbsd or independently building them. I might want to play with freebsd one day but I feel like it might have limited software or I might need to compile a lot of things as compared to archlinux or even debian etc.
    - myaccountonhn 11mo ago
      They're quite different, but in subtle ways, and sometimes it's not about what they offer but rather what they don't offer. What they all have in common is that they have great documentation, and they're all very unix-y and all have some virtualization support for linux. Some things FreeBSD offer are good performance, IME the largest package repository, zfs integration, capsicum and jails. The experience IMO is a bit like Arch, except you'll rely less on a wiki and more on their official documentation. OpenBSD and NetBSD offer a more "complete" experience like you'd expect from a desktop OS, while at the same time being quite minimal and simple. OpenBSD focuses a lot on security, and is probably more likely to work with your hardware with fast wifi. It doesn't offer the same virtualization nor security the same type of security as Linux, but instead encourages you to pledge and unveil your applications, which means you lock down each application by saying what it can and can't do. It notoriously removes features from the kernel that are unmaintained and also doesn't include things it considers insecure like bluetooth. It's probably the OS that gives you the most "works out-of-the-box" experience. NetBSD gives you a full desktop experience as well, but also has some unique security features like kauth, veriexec, extra-hardened chroot and security.curtain. It also offers some cool features like rumpkernels, smolbsd, and is probably the easiest to hack and compile yourself (even cross-compiling is very simple). It has a package manager, pkgsrc, that runs on many other OSes and it's also quite easy to port to new platforms. It's a fun OS to tinker with.
dsp_person 11mo ago
It's possible to run a full DE bwrapped. bwrap --dev-bind /dev /dev --tmpfs /tmp -- labwc bwrap --dev-bind /dev /dev --tmpfs /tmp -- kwin_wayland konsole bwrap --dev-bind /dev /dev --tmpfs /tmp -- startplasma-wayland eglgears_wayland and nvtop show gpu works. That's obviously super permissive, but from there can take things away to expose as much or little of the host system as needed. For my system I'm working on making a few bwrapped "zones". E.g. start a terminal that can see a certain subset of files, and can configure whether it should use gpu or not, have internet or not, have access to local network or not, etc. A bit more project/environment focused than bwrapping programs one by one.

103 points by jaypatelani 11mo ago

jmclnx 11mo ago

Always nice to see NetBSD posts here, that nice OS gets hardly any press.

lovegrenoble 11mo ago

bubblewrap popping )) https://brainteaser.top/bubblespop.html https://brainteaser.top/bubblespop.html

aborsy 11mo ago

I experimented bubblewrap as a better alternative to firejail. Unfortunately there are scripts for few applications. It’s tool for developers. Users are better off flatpaks powered by bubblewrap .

johnisgood 11mo ago
I have been using firejail for a long time now. It seem to work well for me. At times I have to write my own profiles, yeah.
- aborsy 11mo ago
  The situation with firejail is different. There are a lot of profiles, and they are easier to customize and write. But firejail needs root privilege, so there are arguments that it may actually reduce the security, because even though it sandboxes some applications, non-privilege processes may gain privilege if there is a flaw in the fire jail itself. Firejail is also criticized for complexity.
  - johnisgood 11mo ago
    Firejail does not need root privileges at all. You are probably referring to SUID. It is a "setuid ELF 64-bit" executable. I run it with my own user only. Complexity? It just supports a lot of stuff. See the manual page. It seems simple enough to me.
    - aborsy 11mo ago
      Yeah, I meant the same thing! If you run it under your non-root user, and there is a vulnerability in firejail, the jailed process could run as root, right? Complexity referred to large code base and attack surface. I haven’t tried it myself. I would be interested knowing, whether it might lower the security in some cases. Maybe it’s useful to sandbox apps that don’t have flatpaks.
      - jonway 11mo ago
        Yeah, it runs with escalated privs. People wager that SUID binaries are extremely risky and this seems totally reasonable to me! I don't know where to find a really good security analysis of the risks, however. Firejail is most likely not bullet-proof, although not soft by any stretch of the imagination and I don't know how often and how many security researchers are profiling this tool. My gut sense is that flatpak gets much more scrutiny since it ships, and firejail is typically not shipped, but another package as far as I can tell (maybe in some specialized distros?).
      - neobrain 11mo ago
        SUID by itself is not a security liability. Typically, programs using SUID quickly drop their additional privileges during startup to avoid the scenario you're describing. So the vulnerability in firejail would have to be early during startup before any of that. Still, it can happen of course, particularly with memory-unsafe languages. Conversely without root privileges you have to sacrifice strength of your sandbox, widening the vulnerability window in the app you were trying to constrain to begin with. Meanwhile bubblewrap relies on user namespaces, which come with their own set of security problems. They are disabled by default in many distributions for that reason.
Imustaskforhelp 11mo ago
I wanted to build flatpaks cli applications and I still don't understand how to build a flatpak cli in the first place and I have asked in some communities and they mention things like vim being in flatpak but I more so need a definitive guide on how to publish a cli application to flatpak A lot of focus in flatpak sadly gets focused on guis and not clis Like how much efforts would it take to convert a golang static binary/project cli to flatpak?

qwertox 11mo ago

Bubblewrap is a really interesting project, really worth checking out. https://github.com/containers/bubblewrap https://github.com/containers/bubblewrap It's the base for Flatpack, the thing that makes Flatpack be sandboxed. I use it to run Claude Code / Codex / Gemini CLI, to make sure that they have a limited / fake view of my system. You can bind directories into it, or overlay them into it, restrict other kinds of access. If Docker is a thing in a scale between a VM and your OS, Bubblewrap is a thing a scale between Docker and your OS. You use your OS, instead of installing and managing an OS like you do with Docker, but you get filesystem and process isolation like with Docker. Though I had an issue where I cannot use `--new-session`, which is kind of dangerous to not use, but you can get around it if you use seccomp to block ioctl calls, and ptrace.

udev4096 11mo ago
bubblewrap escapes are not unheard of. Infact, it's a common theme that the general linux landscape lacks strong sandboxing, even if you use bwrap, firejail, etc. Especially linux desktop, a security firehazard to say the least unless you are using QubesOS
globular-toast 11mo ago
I documented what I came up with for Claude Code: https://blog.gpkb.org/posts/ai-agent-sandbox/ https://blog.gpkb.org/posts/ai-agent-sandbox/ However, I couldn't get this to work for Codex, it kept failing at the auth bit and I couldn't figure out how to fix it. Anyone got a working solution for Codex?
aborsy 11mo ago
A question. How do you know what permissions are required by an application, to write a bubblewrap script? In AppArmor, you exercise the application and aa-logprof suggests permissions requested by application. If you know AppArmor, usually you can refine those suggestions and write a profile. It may not be ideal, as aa-logprof’s permissions are multiple choice suggestions, require user knowledge and may be too broad or specific, but it could work. You will see that there are many and all kinds of permissions, and there is no way that you will be able to guess them without aa-logprof. What is the equivalent of aa-logprof in bubblewrap and how do you find the required permissions?

DeathArrow 11mo ago

I would have loved to see something like OCI containers.

yjftsjthsd-h 11mo ago
This would underpin that, right? First you build the namespace primitives, then you put the handy wrapper on top that composes a root fs and spawns a container from it using those namespace primitives

matesz 11mo ago

As a side note NetBSD is THE ONLY operating system of which binaries are fully bit by bit reproducible. Moreover it vendores patched gcc compiler, so the entire toolchain is reproducible too as well. How cool is that?! For the record golang brings it further - its entire package registry containing +40mln packages is bit by bit reproducible.

ksec 11mo ago
I thought FreeBSD has it as well with v15?
- Imustaskforhelp 11mo ago
  I think I have read something like that similar as well. Freebsd seems like an amazing project too. I am always confused tho by what is the actual difference b/w freebsd,netbsd,openbsd as I feel like freebsd is taking a lot of positive ideas from both netbsd and openbsd or independently building them. I might want to play with freebsd one day but I feel like it might have limited software or I might need to compile a lot of things as compared to archlinux or even debian etc.
  - myaccountonhn 11mo ago
    They're quite different, but in subtle ways, and sometimes it's not about what they offer but rather what they don't offer. What they all have in common is that they have great documentation, and they're all very unix-y and all have some virtualization support for linux. Some things FreeBSD offer are good performance, IME the largest package repository, zfs integration, capsicum and jails. The experience IMO is a bit like Arch, except you'll rely less on a wiki and more on their official documentation. OpenBSD and NetBSD offer a more "complete" experience like you'd expect from a desktop OS, while at the same time being quite minimal and simple. OpenBSD focuses a lot on security, and is probably more likely to work with your hardware with fast wifi. It doesn't offer the same virtualization nor security the same type of security as Linux, but instead encourages you to pledge and unveil your applications, which means you lock down each application by saying what it can and can't do. It notoriously removes features from the kernel that are unmaintained and also doesn't include things it considers insecure like bluetooth. It's probably the OS that gives you the most "works out-of-the-box" experience. NetBSD gives you a full desktop experience as well, but also has some unique security features like kauth, veriexec, extra-hardened chroot and security.curtain. It also offers some cool features like rumpkernels, smolbsd, and is probably the easiest to hack and compile yourself (even cross-compiling is very simple). It has a package manager, pkgsrc, that runs on many other OSes and it's also quite easy to port to new platforms. It's a fun OS to tinker with.

dsp_person 11mo ago

It's possible to run a full DE bwrapped. bwrap --dev-bind /dev /dev --tmpfs /tmp -- labwc bwrap --dev-bind /dev /dev --tmpfs /tmp -- kwin_wayland konsole bwrap --dev-bind /dev /dev --tmpfs /tmp -- startplasma-wayland eglgears_wayland and nvtop show gpu works. That's obviously super permissive, but from there can take things away to expose as much or little of the host system as needed. For my system I'm working on making a few bwrapped "zones". E.g. start a terminal that can see a certain subset of files, and can configure whether it should use gpu or not, have internet or not, have access to local network or not, etc. A bit more project/environment focused than bwrapping programs one by one.