6 ms·

In a previous HN discussion on the topic of rogue Python packages, readers had suggested bubblewrap and firejail for sandboxing. They limit the access a script

In a previous HN discussion on the topic of rogue Python packages, readers had suggested bubblewrap and firejail for sandboxing. They limit the access a script and its packages have to your filesystem and network. I think that's the better approach - just assume all packages are malicious by default. Can't rely on scanners because of the large number of packages and attacks.

hulitu 4y ago
- lovelearning 4y ago
  It's a problem with every open ecosystem where libraries can be downloaded and run. Rust, Golang, Node all have the same problem. That's why I think it's better to assume anything we download is malicious. Stuff like Bubblewrap and Qubes OS seem to be the better approach compared to relying on vulnerability hunters and scanning tools.
  - quickthrower2 4y ago
    Do both and more. When using an unfamiliar package check it’s upload history. How far does it go back? How did I discover the package, do i trust that source? Etc. Unless your code is never going to touch important data or resources, like for example (but not limited to) being used commercially in any vein then you can’t keep it in a padded cell forever.
unnah 4y ago
That's not going to help much if code from the malicious attacker is still going to end up integrated into the software product being built.
- insanitybit 4y ago
  I disagree. I need to write this up in more detail but it helps quite a lot. 1. An attacker who can access your development environment and your production environment is worse than one who can only access your production environment. You might say "but the end goal is prod", but it's not that simple because of (2). 2. We already have very good tooling for isolating services at runtime. Separating them onto different instances, firewall/security groups, limited API keys, docker/ containers, apparmor, selinux, etc. We have a lot of tooling for "a service in production is owned". What we lack is "a library in dev environment is owned". 3. Devs often have more privileges than your services. It's unfortunate but at a lot of companies, perhaps given some lateral movement around dev envs, you'll find SSH keys to production, browser session cookies that give you console access, source code, chat sessions, internal documents, git keys, gpg keys, etc. So I'm actually fine with a tool that sandboxes the build process but leaves open the hole of "but the attacker can patch the binary and execute code in production". That's a huge win.
- lovelearning 4y ago
  Based on my experience, tools like firejail, ebpf, and opensnitch help us keep security in the forefront, train us to verify behavior instead of trusting blindly, and even persuade end users towards that mindset through our installation steps. If we can spot odd behavior during development and eliminate it from our stacks, the product will be more secure for end users too. There was a time when convenience overrode any security doubts in my mind. But now I routinely use these tools to restrict access, monitor, and review runtime behavior.
quickthrower2 4y ago
Do both and more. When using an unfamiliar package check it’s upload history. How far does it go back? How did I discover the package, do i trust that source? Etc. Unless your code is never going to touch important data or resources, like for example (but not limited to) being used commercially in any vein then you can’t keep it in a padded cell forever.
twawaaay 4y ago
The issue is where your tools are supposed to be generally available on a machine or when your application has access to secrets (like keystores, configuration files, log files, etc.) which is pretty much every application.
cortesoft 4y ago
So that means you can never use any package in code that has to handle sensitive data or manipulate the host machine?
- lovelearning 4y ago
  No, it means don't trust it blindly but instead learn techniques to monitor and verify what it does. I use firejail, ebpf, and opensnitch to restrict access, monitor, and verify runtime behavior. Where possible, persuade end users too to be equally careful. The "linux is safe" cliche blinds both us and end users to its obvious security problems, like running every script as the logged-in user with the same level of access. These malware developers know it and rely on it. That's why we need to move everybody towards a restrict-monitor-verify mindset by default.
ashishbijlani 4y ago
Plug: I've been building Packj [1] to address exactly this problem. It offers “audit” as well as “sandboxing” of PyPI/NPM/Rubygems packages and flags hidden malware or "risky” code behavior such as spawning of shell, use of SSH keys, and mismatch of GitHub code vs packaged code (provenance). 1. https://github.com/ossillate-inc/packj https://github.com/ossillate-inc/packj
- mr_mitm 4y ago
  There is also this, although I haven't tested it yet. The approach is interesting though. https://github.com/avilum/secimport https://github.com/avilum/secimport
actually_a_dog 4y ago
I agree, "assume unknown, unaudited packages are malicious" is the ideal stance. However, I would say that a simple scanning approach could probably take you pretty far. For instance, if you're not using the requests module or the socket module, chances are pretty good there's no data exfiltration going on. It's absolutely not a foolproof approach, but it is a lightweight layer that can be used in a "defense in depth" approach.
- 7373737373 4y ago
  In Python, dynamic imports exist, making this impossible
  - actually_a_dog 4y ago
    I don't see how having dynamic imports matters if all you want to do is detect if a specific file is imported. Run the install and see what gets imported. That's it.
    - 7373737373 4y ago
      If you actually have to execute a program (but have no safe way of doing so), to see if a complex routine that may return any filename imports a safe file or not, then you are facing up against https://en.wikipedia.org/wiki/Rice%27s_theorem https://en.wikipedia.org/wiki/Rice%27s_theorem
      - actually_a_dog 4y ago
        So? Any method of detecting a "malicious package" faces Rice's theorem, unless you want to claim that "malicious" is a trivial property.
        7373737373 4y ago
        Which is why any approaches relying on identity verification or scanning are bound to fail - sandboxing/capability security MUST become built into languages
chlorion 4y ago
Another good option is to create a new user and run everything under the new UID. Running under a new UID has less chances of accidentally leaving something exposed that can allow for sandbox escape. If you run everything from the new UID, it will mostly be contained to it's own $HOME directory and be unable to modify your user's files or system files. Some distros do not protect home directories from being read so it might be worth setting your actual user's $HOME to umask 0700 or whatever. If you are using bwrap while running X11 and not running the sandbox with a new UID, the sandboxed processes may be able to escape via the X11 socket! This can happen even when you don't mount the X11 socket into the sandbox (see abstract sockets)! I think unsharing the network namespace fixes this specific issue (not 100% sure), but there are probably more subtle footguns like this. I really suggest running Wayland with XWayland disabled, and the Wayland socket protected from the sandbox if you want to use bwrap for security purposes!
- lovelearning 4y ago
  Good insights, thank you!
- Quentak 4y ago
  How do you escape the sandbox through a Wayland or X11 socket? Do you have specific code examples? Is there no way to safely run graphical applications in a bwrap sandbox? I thought Wayland was supposed to be better about this.
  - varajelle 4y ago
    I think Wayland is fairly safe, but any X11 client can take screenshots or listen to the keyboard, or emit keyboard event, without limitations.
  - chlorion 4y ago
    I do not have a specific code example, but you can use the normal X11 client interfaces to interact with the X server, which allows a lot of dangerous things such as sending events to other clients. We can imagine a rouge X11 client spawning a terminal and entering text through a virtual input interface, to run an arbitrary command for example. On Wayland, assuming you don't have XWayland enabled and running, it depends on the specific compositor you are using and what Wayland protocols it supports. Sandboxing GUI stuff on Wayland requires at the very least not having XWayland running, and also requires understanding what the compositor allows clients to do by default. Some compositors may have permission dialogues that prevent clients from doing stuff that you didn't expect.
heretoo 4y ago
Another options is "nsjail". I landed on it having considered at firejail, apparmour, selinux, bubblewrap.
- 77pt77 4y ago
  What are the advantages? I'm very frustrated with firejail since I can't for example block execution in my home directory, with the exception of one subdirectory. It just can't be done.
berkes 4y ago
Ive contracted on a project where another, IMO better, solution was employed. This was ruby. The only allowed package (gem) server was one ran by the project. This package-server scanned, vetted and manually checked any version of a lib before "publishing it". If you wanted to e.g. upgrade a package, you'd have to do this on this server first. It would then go through some steps, -automatic scanning, risk analysis, sometimes even needing the eyes of someone from a security team. After that the package was published on this server, and you could pull it onto your dev machine and use it in CI/staging/test/prod etc. Similar steps, to get a new package listed. IMO this is better, because it stops supply-chain attacks before they hit your code, not after they've (potentially) infected the system. Edit: for clarity "only allowed package" wasn't enforced very strictly. A linter and CI would catch any changes to code that would want to fetch packages from elsewhere. It wasn't to protect against rogue developers, but against "stupid me, accidentally upgrading to a version that is infected" and such.

hulitu 4y ago

lovelearning 4y ago
It's a problem with every open ecosystem where libraries can be downloaded and run. Rust, Golang, Node all have the same problem. That's why I think it's better to assume anything we download is malicious. Stuff like Bubblewrap and Qubes OS seem to be the better approach compared to relying on vulnerability hunters and scanning tools.
- quickthrower2 4y ago
  Do both and more. When using an unfamiliar package check it’s upload history. How far does it go back? How did I discover the package, do i trust that source? Etc. Unless your code is never going to touch important data or resources, like for example (but not limited to) being used commercially in any vein then you can’t keep it in a padded cell forever.

unnah 4y ago

That's not going to help much if code from the malicious attacker is still going to end up integrated into the software product being built.

insanitybit 4y ago
I disagree. I need to write this up in more detail but it helps quite a lot. 1. An attacker who can access your development environment and your production environment is worse than one who can only access your production environment. You might say "but the end goal is prod", but it's not that simple because of (2). 2. We already have very good tooling for isolating services at runtime. Separating them onto different instances, firewall/security groups, limited API keys, docker/ containers, apparmor, selinux, etc. We have a lot of tooling for "a service in production is owned". What we lack is "a library in dev environment is owned". 3. Devs often have more privileges than your services. It's unfortunate but at a lot of companies, perhaps given some lateral movement around dev envs, you'll find SSH keys to production, browser session cookies that give you console access, source code, chat sessions, internal documents, git keys, gpg keys, etc. So I'm actually fine with a tool that sandboxes the build process but leaves open the hole of "but the attacker can patch the binary and execute code in production". That's a huge win.
lovelearning 4y ago
Based on my experience, tools like firejail, ebpf, and opensnitch help us keep security in the forefront, train us to verify behavior instead of trusting blindly, and even persuade end users towards that mindset through our installation steps. If we can spot odd behavior during development and eliminate it from our stacks, the product will be more secure for end users too. There was a time when convenience overrode any security doubts in my mind. But now I routinely use these tools to restrict access, monitor, and review runtime behavior.

quickthrower2 4y ago

Do both and more. When using an unfamiliar package check it’s upload history. How far does it go back? How did I discover the package, do i trust that source? Etc. Unless your code is never going to touch important data or resources, like for example (but not limited to) being used commercially in any vein then you can’t keep it in a padded cell forever.

twawaaay 4y ago

The issue is where your tools are supposed to be generally available on a machine or when your application has access to secrets (like keystores, configuration files, log files, etc.) which is pretty much every application.

cortesoft 4y ago

So that means you can never use any package in code that has to handle sensitive data or manipulate the host machine?

lovelearning 4y ago
No, it means don't trust it blindly but instead learn techniques to monitor and verify what it does. I use firejail, ebpf, and opensnitch to restrict access, monitor, and verify runtime behavior. Where possible, persuade end users too to be equally careful. The "linux is safe" cliche blinds both us and end users to its obvious security problems, like running every script as the logged-in user with the same level of access. These malware developers know it and rely on it. That's why we need to move everybody towards a restrict-monitor-verify mindset by default.

ashishbijlani 4y ago

Plug: I've been building Packj [1] to address exactly this problem. It offers “audit” as well as “sandboxing” of PyPI/NPM/Rubygems packages and flags hidden malware or "risky” code behavior such as spawning of shell, use of SSH keys, and mismatch of GitHub code vs packaged code (provenance). 1. https://github.com/ossillate-inc/packj https://github.com/ossillate-inc/packj

mr_mitm 4y ago
There is also this, although I haven't tested it yet. The approach is interesting though. https://github.com/avilum/secimport https://github.com/avilum/secimport

actually_a_dog 4y ago

I agree, "assume unknown, unaudited packages are malicious" is the ideal stance. However, I would say that a simple scanning approach could probably take you pretty far. For instance, if you're not using the requests module or the socket module, chances are pretty good there's no data exfiltration going on. It's absolutely not a foolproof approach, but it is a lightweight layer that can be used in a "defense in depth" approach.

7373737373 4y ago
In Python, dynamic imports exist, making this impossible
- actually_a_dog 4y ago
  I don't see how having dynamic imports matters if all you want to do is detect if a specific file is imported. Run the install and see what gets imported. That's it.
  - 7373737373 4y ago
    If you actually have to execute a program (but have no safe way of doing so), to see if a complex routine that may return any filename imports a safe file or not, then you are facing up against https://en.wikipedia.org/wiki/Rice%27s_theorem https://en.wikipedia.org/wiki/Rice%27s_theorem
    - actually_a_dog 4y ago
      So? Any method of detecting a "malicious package" faces Rice's theorem, unless you want to claim that "malicious" is a trivial property.
      - 7373737373 4y ago
        Which is why any approaches relying on identity verification or scanning are bound to fail - sandboxing/capability security MUST become built into languages

chlorion 4y ago

Another good option is to create a new user and run everything under the new UID. Running under a new UID has less chances of accidentally leaving something exposed that can allow for sandbox escape. If you run everything from the new UID, it will mostly be contained to it's own $HOME directory and be unable to modify your user's files or system files. Some distros do not protect home directories from being read so it might be worth setting your actual user's $HOME to umask 0700 or whatever. If you are using bwrap while running X11 and not running the sandbox with a new UID, the sandboxed processes may be able to escape via the X11 socket! This can happen even when you don't mount the X11 socket into the sandbox (see abstract sockets)! I think unsharing the network namespace fixes this specific issue (not 100% sure), but there are probably more subtle footguns like this. I really suggest running Wayland with XWayland disabled, and the Wayland socket protected from the sandbox if you want to use bwrap for security purposes!

lovelearning 4y ago
Good insights, thank you!
Quentak 4y ago
How do you escape the sandbox through a Wayland or X11 socket? Do you have specific code examples? Is there no way to safely run graphical applications in a bwrap sandbox? I thought Wayland was supposed to be better about this.
- varajelle 4y ago
  I think Wayland is fairly safe, but any X11 client can take screenshots or listen to the keyboard, or emit keyboard event, without limitations.
- chlorion 4y ago
  I do not have a specific code example, but you can use the normal X11 client interfaces to interact with the X server, which allows a lot of dangerous things such as sending events to other clients. We can imagine a rouge X11 client spawning a terminal and entering text through a virtual input interface, to run an arbitrary command for example. On Wayland, assuming you don't have XWayland enabled and running, it depends on the specific compositor you are using and what Wayland protocols it supports. Sandboxing GUI stuff on Wayland requires at the very least not having XWayland running, and also requires understanding what the compositor allows clients to do by default. Some compositors may have permission dialogues that prevent clients from doing stuff that you didn't expect.

heretoo 4y ago

Another options is "nsjail". I landed on it having considered at firejail, apparmour, selinux, bubblewrap.

77pt77 4y ago
What are the advantages? I'm very frustrated with firejail since I can't for example block execution in my home directory, with the exception of one subdirectory. It just can't be done.

berkes 4y ago

Ive contracted on a project where another, IMO better, solution was employed. This was ruby. The only allowed package (gem) server was one ran by the project. This package-server scanned, vetted and manually checked any version of a lib before "publishing it". If you wanted to e.g. upgrade a package, you'd have to do this on this server first. It would then go through some steps, -automatic scanning, risk analysis, sometimes even needing the eyes of someone from a security team. After that the package was published on this server, and you could pull it onto your dev machine and use it in CI/staging/test/prod etc. Similar steps, to get a new package listed. IMO this is better, because it stops supply-chain attacks before they hit your code, not after they've (potentially) infected the system. Edit: for clarity "only allowed package" wasn't enforced very strictly. A linter and CI would catch any changes to code that would want to fetch packages from elsewhere. It wasn't to protect against rogue developers, but against "stupid me, accidentally upgrading to a version that is infected" and such.