12 ms·

I think a proper way to solve this issue, not specific to python but languages running in a VM in general, would be to have some sort of language support where

I think a proper way to solve this issue, not specific to python but languages running in a VM in general, would be to have some sort of language support where you specifically define what access rights/ system resources you allow for any given dependency. Example of defining project dependencies: { "apollo-client": { "version": "...", "access": ["fetch"] // only fetch allowed }, "stringutils": { "version": "...", "access": [] // no system resources allowed for this dependency, own or transitive }, ... } It would probably require the language to limit monkey-patching core primitives (such as Object.prototype in javascript), and it would be more cumbersome for the developer to define the permissions it gives to each dependency. These required permissions could be listed on the package site (eg npm or PyPI) and the developer would just copy paste the permissions when adding the dependency. But if you upgrade a dependency version and it now requires a permission that seems suspicious (eg "stringutils" needing "filesystem"), it would prompt the developer to stop and investigate, or if it seems justified add the permission to "access" list.

louislang 4y ago
This is one of the projects we're working on (and open sourcing)! Currently allows you to specify allowed resources during the package installation in a way very similar to what you've outlined [1]. The sandbox itself lives here [2] and can be integrated into other projects. 1. https://github.com/phylum-dev/cli/blob/main/extensions/npm/PhylumExt.toml https://github.com/phylum-dev/cli/blob/main/extensions/npm/P... 2. https://github.com/phylum-dev/birdcage https://github.com/phylum-dev/birdcage
- deleted 4y ago
  [deleted]
c0balt 4y ago
At the beginning the permissions aspect of deno[0] was actually on of the major selling points for me. The approach used there was to begin at zero and offer granular permission control, e.g. `--allow-read=data.csv`, for filesystem, network etc. I would love to have this for, e.g., python or npm packages. [0]: https://deno.land/manual@v1.27.0/getting_started/permissions https://deno.land/manual@v1.27.0/getting_started/permissions
- jens0 4y ago
  Doesn't this only apply to the entire process? Not the individual dependencies, right? Just confirming, Deno was my first thought with this, it requires the developer to deliberately enable permissions needed.
  - SahAssar 4y ago
    Yes, it applies to the whole process. It's incredibly hard to sandbox dependencies individually since you don't know how your code or other dependencies interact with it. If you want you can run dependencies in a worker process and sandbox that tighter, but that is quite a bit of work.
    - ashishbijlani 4y ago
      That’s exactly why I use strace-based to sandbox ALL dependencies: https://github.com/ossillate-inc/packj https://github.com/ossillate-inc/packj
    - juancampa 4y ago
      This is exactly what I've done for Membrane[0]. It's capabilities based, even to get the time (and thus introduce non-determinism) you need a capability. Dependencies run as separate processes and everything is orthogonally persistent. It's a typescript/javascript system for personal automation built entirely within VSCode. Stay tuned, I'll be posting a video this week. [0] https://membrane.io https://membrane.io
      - uhhyeahdude 4y ago
        Hi there! I just wanted to let you know that I read the blog posts, and membrane sounds extremely cool. Ambitious, though. If you don’t mind a small bit of feedback: it would be encouraging to potential users or testers to see some semi-regular posts related to development. It would also be great to see how membrane might work to build a tool using current APIs. I know there is video forthcoming, and perhaps this will be addressed. I didn’t look at the GitHub, where I probably could glean some additional info. But from my perspective, a development blog builds trust and anticipation. It’s also a great way to check your assumptions (or have them checked, rather). Good luck with the project. I hope it delivers, because I’d love to use it. Signed up for the mailing list.
berniedurfee 4y ago
Yep, a declarative mechanism would be nice like OAuth scopes. Though, like scopes, I think many times packages would need broad access, but maybe not?
rileymat2 4y ago
I thought that Java Applets (and maybe flash, I am less familiar) had an advanced security model, but it was exploit after exploit because of the huge attack surfaces? I suspect you may run into similar sandbox escapes once things are complicated enough. So it seems like a good idea if they can be made bug free, but good luck with that?
- insanitybit 4y ago
  Part of the problem with the Java sandbox is that it was enforced entirely by the VM + the VM is written in C++. The idea is not inherently bad.
  - nl 4y ago
    It's been a while since I worked in this area but my recollection was that most JVM security issues in this areas were bypasses of the Java Security Manager often by confusing it about code origin. That's all Java code, not C++.
    - insanitybit 4y ago
      It's been so long I could be remembering incorrectly.
    - int_19h 4y ago
      For both Java and .NET, there were actual verification bugs, as well - when bytecode that's not supposed to be valid gets past the verifier and results in e.g. mistyped references (which can then be used for all kinds of creative vtable abuse). Sometimes it can even be a bug in the VM spec itself, because implications of two different features interacting weren't fully considered. But, yes, we've tried this idea many times now, and it never held up for long.
- erustemi 4y ago
  Maybe the issue was with how powerful and unrestricted reflection was in java before introduction of modules.
jjav 4y ago
This is essentially the fine-grained control the Java Security Manager enabled. But hardly anyone used it and it was deprecated sadly.
- charleslmunger 4y ago
  And it didn't really work in practice. Every callback or thread hop is a security vulnerability by default.
uncletammy 4y ago
Your proposed solution sounds an awful lot like a manifest file https://en.wikipedia.org/wiki/Manifest_file https://en.wikipedia.org/wiki/Manifest_file
insanitybit 4y ago
I've been messing around with some ideas. 1. `autobox` (to be renamed lol) [0]. It's basically a Rust interpreter that performs taint and effect analysis, reporting on both, allowing you to use that information to generate sandboxes. ie: "autobox sees you used the string '~/.config' to read a file, and that is all the IO performed, so that is all the IO you get". 2. I'm working on a container based `cargo` with `riff` built in that aims to work for the vast majority of projects and sandbox your build with a defined threat model. The goal is to be able to basically `alias cargo=cargo-sandboxed` and have the same experience but with a restricted container environment + better auditing of things happening in the container. 3. I previously built a POC of a `Sandbox.toml` and `Sandbox.lock` with a policy language that allowed you to specify a policy for a given build step. Unfortunately, I couldn't decide on how I wanted it to work in terms of "do I generate a single sandbox for the entire build, or do I run each build stage in its own sandbox" - there are tradeoffs for both. Here's a lil snippet: [build-permissions.file-system] // All paths are relative to the project directory unless they start with `/` "../" = {permissions = ["read"]} // "$target" being a special path "$target" = {permissions = ["read", "write"]} // Source this path from the environment at build time, `optional` means it's // ok if it isn't available "$env::PROTOC_PATH" = {permissions = ["read", "execute"], optional=true} // Default protobuf installation paths, via regex "^(/usr)?/bin/protoc" = {permissions = ["read", "execute"], regex=true} Once I'm done with (2) though I think I'll tackle (3). `autobox` is fun but I think it may be impractical without more language level support and no matter what I'd end up having to implement it in the compiler at some point, which means it would be unusable without nightly or a fork. I'm going to try to wrap up an autobox POC that handles branching and loops, publish it, and see if someone who does more compilery things is willing to pick it up. As for (2) and (3) I believe I can build practical implementations for both. [0] https://github.com/insanitybit/autobox/ https://github.com/insanitybit/autobox/
- louislang 4y ago
  This is really cool work! Also a fan of Grapl.
  - insanitybit 4y ago
nicoty 4y ago
I watched a video[0] about the Roc language recently, and they do something interesting to address this: they have a layer in their language called "platforms" and the idea behind these are that there are many different platforms that you can choose between to run code with and each one has different permissions. So one platform might be sandboxed and disallow the use of certain unsafe APIs whereas another might be less sandboxed. [0] https://m.youtube.com/watch?v=cpQwtwVKAfU https://m.youtube.com/watch?v=cpQwtwVKAfU
nijave 4y ago
I saw a similar proposal (I think with JavaScript/node) not too long ago that deacribed limiting packages to data in their own namespace. For instance third-party-dep-a would only have access to data it created or was passed in versus indiscriminately accessing anything in the language VM. Even this would be a good step in the right direction although you'd likely still need something like you e described for accessing shared system resources (aka the mobile phone security model)
Legogris 4y ago
For JS, You are basically talking about Lavamoat. It provides tooling and policies for SES, which aims to make it into standards. https://github.com/LavaMoat/LavaMoat https://github.com/LavaMoat/LavaMoat https://github.com/endojs/endo/tree/master/packages/ses https://github.com/endojs/endo/tree/master/packages/ses
dheera 4y ago
Another thing I think might help is (a) Discourage any future use of ">=" in version dependencies. Specify an exact version. That way a future compromised version doesn't get pulled (b) Every build system needs better ways of having multiple versions of a same dependency coexist. I should be able to have one of my project's dependencies depend on "numpy==1.15" and another dependency depend on "numpy==1.16" and they should be able to coexist in the SAME environment and "see" exactly the numpy versions they requested. For python we should think about how to support something like this in the future: import numpy==1.15 and have it just work. That way if a hacker compromises PyPI and releases a malicious numpy 1.19 it won't get pulled in accidentally. Here's a bit of a joke I made before that might be an interesting starting point, though since it uses virtualenv behind the hood it doesn't have a way for multiple versions of one package to exist. I don't think it's impossible to do though with some additional work. https://github.com/dheera/magicimport.py https://github.com/dheera/magicimport.py Sample code: from magicimport import magicimport tornado = magicimport("tornado", version = "4.5")
- cozzyd 4y ago
  Would you run into dynamic linker problems in this case due to symbol conflicts? Or does symbol versioning magically resolve that somehow?
  - dheera 4y ago
    Prefix all symbols with versions? foo() becomes v1_15_4_foo() automatically
- MikePlacid 4y ago
  > I should be able to have one of my project's dependencies depend on "numpy==1.15" and another dependency depend on "numpy==1.16" and they should be able to coexist in the SAME environment Now I see what people at one respectable, big project were thinking when they allowed 7 different versions of OpenSSL to be statically linked to the same executable… Seriously, this idea may save you from some not very interesting work, but it will create the need of much bigger amount of work which while potentially interesting is not very productive. You are toying with exponential growth here, like a chain reaction - like a bomb.
rollcat 4y ago
Check out OpenBSD's pledge(2): https://man.openbsd.org/pledge.2 https://man.openbsd.org/pledge.2 It does exactly that (although on a per-process basis). I don't think this kind of permission system can be retrofitted into an existing language without direct OS support, and probably not at the library level (you'd need something like per-page permissions which would get hairy real fast).
- musicale 4y ago
  I like this. I'd try to keep the permission sets as small, limited, and simple as possible though.
  - rollcat 4y ago
    > I'd try to keep the permission sets as small and simple as possible though. You've described OpenBSD in general. I recommend a deeper dive - it's fantastically refreshing, how simple yet functional an OS can be.
    - paulmd 4y ago
      Same for FreeBSD. Incredibly code-stable and well-documented by Linux standards. I often use the FreeBSD Handbook as an example of first-party documentation done right, and that's only possible because of deliberately limited "churn for churn's sake". The kinds of regular code rot and attrition that Linux suffers just does not take place on BSD systems because if you contribute something new you're expected to make your thing mesh with the ecosystem, they don't tolerate the "I'm gonna churn everyone's environments because my version is 10% better than the existing one" that tends to take place on Linux. As an example, the amount of init systems that Ubuntu has gone through over the last 20 years is completely insane by BSD standards. They've gone from sysvinit to upstart to systemd. You don't have people ripping out the graphics and audio subsystems and doing total rewrites, either. It's nuts that a lot of Linux people don't even realize that it doesn't have to be that way - that's not "just how maintenance is supposed to work", that's a bunch of incredibly bad behavior on the parts of distros and Poettering specifically that has become highly normalized and overlooked. Maintenance shouldn't be breaking things like that. "we never break userland" doesn't have to be a suicide pact either - BSD has actually maintained a much more stable ABI than the linux kernel without crystallizing a bunch of shitty bug-compatibility stuff like Linux is doing either. It's all insanely stable and competent and professional by linux standards. If I was developing a product for true long-term support, with the minimum possible engineering effort devoted to fighting churn, BSD would be at the top of the list.
ransom1538 4y ago
It's only a matter of time, until, someone with some cash + a good connection to a package just does what is going to happen.
p1necone 4y ago
You could remove the need to explicitly specify the permissions somewhat by enforcing semver for permission addition. If you own a package at version 1.X.X and you want to add a permission requirement you have to bump the version to 2.0.0. If you also allow people to opt in to a less strict "auto allow all the currently required permissions for these dependencies" mode, they would at least know for sure nothing can touch anything new unless they explicitly bump the major version. If you're extra concerned about security you can explicitly specify them so it's really obvious when a major version bump adds new ones, but it removes some of the friction.
e1g 4y ago
Node.js is building something very similar: Permission Model https://github.com/nodejs/security-wg/issues/791 https://github.com/nodejs/security-wg/issues/791
ciupicri 4y ago
There were some attempts made for Python 10 - 15 years ago and the conclusions were that it's very hard to do it right. If I remember correctly Zope was using some sandboxing and because of it, it took a while time to catch up with newer versions of Python. You had to compile you own Python because the one that came with the Linux distribution was too new. Also If I'm not mistaking I think that PyPy has some sandboxing. Anyway I'll leave this from 2013 here: > After having work during 3 years on a pysandbox project to sandbox untrusted code, I now reached a point where I am convinced that pysandbox is broken by design. Different developers tried to convinced me before that pysandbox design is unsafe, but I had to experience it myself to be convineced. > It would also be nice to help developers looking for a sandbox for their application. Please tell me if you know sandbox projects for Python so I can redirect users of pysandbox to a safer solution. I already know PyPy sandbox. -- https://mail.python.org/pipermail/python-dev/2013-November/130132.html https://mail.python.org/pipermail/python-dev/2013-November/1...
0xblinq 4y ago
Wouldn't running your development environment inside docker provide the same safety levels?
- louislang 4y ago
  Containers aren't great security boundaries. To get the safety you'd really need, you should absolutely use a VM.
ryukafalz 4y ago
I broadly agree, but if I'm reading your suggestion correctly I think that "access" list is too coarse-grained still! It looks like you're suggesting a predefined list of permissions that can be granted to a dependency, but... why not go even further? If the list of things you're passing in are references rather than strings, then you could do... const apollo = require('apollo-client', {fetch}); const stringutils = require('stringutils'); That way, you can pass in at a very fine-grained level any object you want to, including things like `fetch` and `fs`. But then, you could just as easily pass in a custom `loggedFetch` module like so, that wraps `fetch` and logs all network requests made: const apollo = require('apollo-client', {fetch: loggedFetch}); This is the object-capability security model, and as a sibling commenter pointed out, LavaMoat basically does this for JavaScript dependencies. (This is also basically just lexical scope, by the way, if your language is strict enough - though most unfortunately aren't.)
- FractalHQ 4y ago
  I love this idea. But what about peer deps? And their dependencies? What if multiple deps share peer deps? Is there a simple way permissions could be resolved in the web of modules? Also- tangential but I really wish people would stop using require and use ESM whenever possible. I cringe when I see `require` just as much as when I see `var` being used in 2022, but I’m not certain there’s never a good reason not to use ESM.
  - nirimda 4y ago
    If you want to override the dependency of your dependency, then it's your dependency too - so if you were using the javascript approach up there you might have var bar = require("bar", {baz: loggingBaz}) var foo = require("foo", {bar}) This is true even if you never otherwise use bar. This is certainly how peer dependencies work in nix flakes, for instance, where you can say something like bar = //.... bar.inputs.baz.follows = loggingBaz; foo = //.... foo.inputs.bar.follows = bar; Since nix flakes is a package/dependency manager, it generates a lockfile that you could inspect too see what all the dependencies are, and make sure you got it right (oh, I still have two bar's, I must have forgotten to override some other dep). I suppose any compiler that involves a linking stage would in principle be able to generate some comparable output at the language level. (Apologies for using require and var, but I'm convinced the functional syntax is semantically clearer in this context.)
bernawil 4y ago
A resource access model for dependencies doesn't make much sense to me, there's basically only 2 things you want to gate access for libraries: filesystem and network. And it's all-in. A library that needs network access may be legit today and after an update start exfiltrating data to a different url. It seems easier to grep for fs and network calls in the library code than any of that.
- jononor 4y ago
  Restricting a process to only be able to access opt-in list of directories underneath the project directory would be useful. Assuming one uses a venv, and all the dependencies are contained there. Then one might want some data folder. And have at least prevented dels from scraping user-wide secrets.
  - bernawil 4y ago
    You're describing a chroot jail. key there is "process". Dealing with processes permissions is the OS's job. A if a language wants to deal in library's security it should strive to make static analysis possible. Eg: the language guarantees that network and filesystem calls can only be done with a single function, statically so I can audit that leftpad indeed doesn't make network calls .
mdavidn 4y ago
In a post-Spectre and post-rowhammer world, I have my doubts that any sub-process security boundary will prove durable in the long run.
marmada 4y ago
Wasmtime / WASI does this extremely well.
hayley-patton 4y ago
Newspeak? https://newspeaklanguage.org/ https://newspeaklanguage.org/
pjmlp 4y ago
Java and .NET had it, and in both cases they eventually dropped it, because many security exploits were cause by developers not really understanding how to use them. In the end they became yet another attack vector, and now everyone should use OS security services instead.
lionkor 4y ago
Sort of like BSD's pledge() and similar APIs[1] [1]: https://man.openbsd.org/pledge.2 https://man.openbsd.org/pledge.2

louislang 4y ago

This is one of the projects we're working on (and open sourcing)! Currently allows you to specify allowed resources during the package installation in a way very similar to what you've outlined [1]. The sandbox itself lives here [2] and can be integrated into other projects. 1. https://github.com/phylum-dev/cli/blob/main/extensions/npm/PhylumExt.toml https://github.com/phylum-dev/cli/blob/main/extensions/npm/P... 2. https://github.com/phylum-dev/birdcage https://github.com/phylum-dev/birdcage

deleted 4y ago
[deleted]

c0balt 4y ago

At the beginning the permissions aspect of deno[0] was actually on of the major selling points for me. The approach used there was to begin at zero and offer granular permission control, e.g. `--allow-read=data.csv`, for filesystem, network etc. I would love to have this for, e.g., python or npm packages. [0]: https://deno.land/manual@v1.27.0/getting_started/permissions https://deno.land/manual@v1.27.0/getting_started/permissions

jens0 4y ago
Doesn't this only apply to the entire process? Not the individual dependencies, right? Just confirming, Deno was my first thought with this, it requires the developer to deliberately enable permissions needed.
- SahAssar 4y ago
  Yes, it applies to the whole process. It's incredibly hard to sandbox dependencies individually since you don't know how your code or other dependencies interact with it. If you want you can run dependencies in a worker process and sandbox that tighter, but that is quite a bit of work.
  - ashishbijlani 4y ago
    That’s exactly why I use strace-based to sandbox ALL dependencies: https://github.com/ossillate-inc/packj https://github.com/ossillate-inc/packj
  - juancampa 4y ago
    This is exactly what I've done for Membrane[0]. It's capabilities based, even to get the time (and thus introduce non-determinism) you need a capability. Dependencies run as separate processes and everything is orthogonally persistent. It's a typescript/javascript system for personal automation built entirely within VSCode. Stay tuned, I'll be posting a video this week. [0] https://membrane.io https://membrane.io
    - uhhyeahdude 4y ago
      Hi there! I just wanted to let you know that I read the blog posts, and membrane sounds extremely cool. Ambitious, though. If you don’t mind a small bit of feedback: it would be encouraging to potential users or testers to see some semi-regular posts related to development. It would also be great to see how membrane might work to build a tool using current APIs. I know there is video forthcoming, and perhaps this will be addressed. I didn’t look at the GitHub, where I probably could glean some additional info. But from my perspective, a development blog builds trust and anticipation. It’s also a great way to check your assumptions (or have them checked, rather). Good luck with the project. I hope it delivers, because I’d love to use it. Signed up for the mailing list.

berniedurfee 4y ago

Yep, a declarative mechanism would be nice like OAuth scopes. Though, like scopes, I think many times packages would need broad access, but maybe not?

rileymat2 4y ago

I thought that Java Applets (and maybe flash, I am less familiar) had an advanced security model, but it was exploit after exploit because of the huge attack surfaces? I suspect you may run into similar sandbox escapes once things are complicated enough. So it seems like a good idea if they can be made bug free, but good luck with that?

insanitybit 4y ago
Part of the problem with the Java sandbox is that it was enforced entirely by the VM + the VM is written in C++. The idea is not inherently bad.
- nl 4y ago
  It's been a while since I worked in this area but my recollection was that most JVM security issues in this areas were bypasses of the Java Security Manager often by confusing it about code origin. That's all Java code, not C++.
  - insanitybit 4y ago
    It's been so long I could be remembering incorrectly.
  - int_19h 4y ago
    For both Java and .NET, there were actual verification bugs, as well - when bytecode that's not supposed to be valid gets past the verifier and results in e.g. mistyped references (which can then be used for all kinds of creative vtable abuse). Sometimes it can even be a bug in the VM spec itself, because implications of two different features interacting weren't fully considered. But, yes, we've tried this idea many times now, and it never held up for long.
erustemi 4y ago
Maybe the issue was with how powerful and unrestricted reflection was in java before introduction of modules.

jjav 4y ago

This is essentially the fine-grained control the Java Security Manager enabled. But hardly anyone used it and it was deprecated sadly.

charleslmunger 4y ago
And it didn't really work in practice. Every callback or thread hop is a security vulnerability by default.

uncletammy 4y ago

Your proposed solution sounds an awful lot like a manifest file https://en.wikipedia.org/wiki/Manifest_file https://en.wikipedia.org/wiki/Manifest_file

insanitybit 4y ago

I've been messing around with some ideas. 1. `autobox` (to be renamed lol) [0]. It's basically a Rust interpreter that performs taint and effect analysis, reporting on both, allowing you to use that information to generate sandboxes. ie: "autobox sees you used the string '~/.config' to read a file, and that is all the IO performed, so that is all the IO you get". 2. I'm working on a container based `cargo` with `riff` built in that aims to work for the vast majority of projects and sandbox your build with a defined threat model. The goal is to be able to basically `alias cargo=cargo-sandboxed` and have the same experience but with a restricted container environment + better auditing of things happening in the container. 3. I previously built a POC of a `Sandbox.toml` and `Sandbox.lock` with a policy language that allowed you to specify a policy for a given build step. Unfortunately, I couldn't decide on how I wanted it to work in terms of "do I generate a single sandbox for the entire build, or do I run each build stage in its own sandbox" - there are tradeoffs for both. Here's a lil snippet: [build-permissions.file-system] // All paths are relative to the project directory unless they start with `/` "../" = {permissions = ["read"]} // "$target" being a special path "$target" = {permissions = ["read", "write"]} // Source this path from the environment at build time, `optional` means it's // ok if it isn't available "$env::PROTOC_PATH" = {permissions = ["read", "execute"], optional=true} // Default protobuf installation paths, via regex "^(/usr)?/bin/protoc" = {permissions = ["read", "execute"], regex=true} Once I'm done with (2) though I think I'll tackle (3). `autobox` is fun but I think it may be impractical without more language level support and no matter what I'd end up having to implement it in the compiler at some point, which means it would be unusable without nightly or a fork. I'm going to try to wrap up an autobox POC that handles branching and loops, publish it, and see if someone who does more compilery things is willing to pick it up. As for (2) and (3) I believe I can build practical implementations for both. [0] https://github.com/insanitybit/autobox/ https://github.com/insanitybit/autobox/

louislang 4y ago
This is really cool work! Also a fan of Grapl.
- insanitybit 4y ago

nicoty 4y ago

I watched a video[0] about the Roc language recently, and they do something interesting to address this: they have a layer in their language called "platforms" and the idea behind these are that there are many different platforms that you can choose between to run code with and each one has different permissions. So one platform might be sandboxed and disallow the use of certain unsafe APIs whereas another might be less sandboxed. [0] https://m.youtube.com/watch?v=cpQwtwVKAfU https://m.youtube.com/watch?v=cpQwtwVKAfU

nijave 4y ago

I saw a similar proposal (I think with JavaScript/node) not too long ago that deacribed limiting packages to data in their own namespace. For instance third-party-dep-a would only have access to data it created or was passed in versus indiscriminately accessing anything in the language VM. Even this would be a good step in the right direction although you'd likely still need something like you e described for accessing shared system resources (aka the mobile phone security model)

Legogris 4y ago

For JS, You are basically talking about Lavamoat. It provides tooling and policies for SES, which aims to make it into standards. https://github.com/LavaMoat/LavaMoat https://github.com/LavaMoat/LavaMoat https://github.com/endojs/endo/tree/master/packages/ses https://github.com/endojs/endo/tree/master/packages/ses

dheera 4y ago

Another thing I think might help is (a) Discourage any future use of ">=" in version dependencies. Specify an exact version. That way a future compromised version doesn't get pulled (b) Every build system needs better ways of having multiple versions of a same dependency coexist. I should be able to have one of my project's dependencies depend on "numpy==1.15" and another dependency depend on "numpy==1.16" and they should be able to coexist in the SAME environment and "see" exactly the numpy versions they requested. For python we should think about how to support something like this in the future: import numpy==1.15 and have it just work. That way if a hacker compromises PyPI and releases a malicious numpy 1.19 it won't get pulled in accidentally. Here's a bit of a joke I made before that might be an interesting starting point, though since it uses virtualenv behind the hood it doesn't have a way for multiple versions of one package to exist. I don't think it's impossible to do though with some additional work. https://github.com/dheera/magicimport.py https://github.com/dheera/magicimport.py Sample code: from magicimport import magicimport tornado = magicimport("tornado", version = "4.5")

cozzyd 4y ago
Would you run into dynamic linker problems in this case due to symbol conflicts? Or does symbol versioning magically resolve that somehow?
- dheera 4y ago
  Prefix all symbols with versions? foo() becomes v1_15_4_foo() automatically
MikePlacid 4y ago
> I should be able to have one of my project's dependencies depend on "numpy==1.15" and another dependency depend on "numpy==1.16" and they should be able to coexist in the SAME environment Now I see what people at one respectable, big project were thinking when they allowed 7 different versions of OpenSSL to be statically linked to the same executable… Seriously, this idea may save you from some not very interesting work, but it will create the need of much bigger amount of work which while potentially interesting is not very productive. You are toying with exponential growth here, like a chain reaction - like a bomb.

rollcat 4y ago

Check out OpenBSD's pledge(2): https://man.openbsd.org/pledge.2 https://man.openbsd.org/pledge.2 It does exactly that (although on a per-process basis). I don't think this kind of permission system can be retrofitted into an existing language without direct OS support, and probably not at the library level (you'd need something like per-page permissions which would get hairy real fast).

musicale 4y ago
I like this. I'd try to keep the permission sets as small, limited, and simple as possible though.
- rollcat 4y ago
  > I'd try to keep the permission sets as small and simple as possible though. You've described OpenBSD in general. I recommend a deeper dive - it's fantastically refreshing, how simple yet functional an OS can be.
  - paulmd 4y ago
    Same for FreeBSD. Incredibly code-stable and well-documented by Linux standards. I often use the FreeBSD Handbook as an example of first-party documentation done right, and that's only possible because of deliberately limited "churn for churn's sake". The kinds of regular code rot and attrition that Linux suffers just does not take place on BSD systems because if you contribute something new you're expected to make your thing mesh with the ecosystem, they don't tolerate the "I'm gonna churn everyone's environments because my version is 10% better than the existing one" that tends to take place on Linux. As an example, the amount of init systems that Ubuntu has gone through over the last 20 years is completely insane by BSD standards. They've gone from sysvinit to upstart to systemd. You don't have people ripping out the graphics and audio subsystems and doing total rewrites, either. It's nuts that a lot of Linux people don't even realize that it doesn't have to be that way - that's not "just how maintenance is supposed to work", that's a bunch of incredibly bad behavior on the parts of distros and Poettering specifically that has become highly normalized and overlooked. Maintenance shouldn't be breaking things like that. "we never break userland" doesn't have to be a suicide pact either - BSD has actually maintained a much more stable ABI than the linux kernel without crystallizing a bunch of shitty bug-compatibility stuff like Linux is doing either. It's all insanely stable and competent and professional by linux standards. If I was developing a product for true long-term support, with the minimum possible engineering effort devoted to fighting churn, BSD would be at the top of the list.

ransom1538 4y ago

It's only a matter of time, until, someone with some cash + a good connection to a package just does what is going to happen.

p1necone 4y ago

You could remove the need to explicitly specify the permissions somewhat by enforcing semver for permission addition. If you own a package at version 1.X.X and you want to add a permission requirement you have to bump the version to 2.0.0. If you also allow people to opt in to a less strict "auto allow all the currently required permissions for these dependencies" mode, they would at least know for sure nothing can touch anything new unless they explicitly bump the major version. If you're extra concerned about security you can explicitly specify them so it's really obvious when a major version bump adds new ones, but it removes some of the friction.

e1g 4y ago

Node.js is building something very similar: Permission Model https://github.com/nodejs/security-wg/issues/791 https://github.com/nodejs/security-wg/issues/791

ciupicri 4y ago

There were some attempts made for Python 10 - 15 years ago and the conclusions were that it's very hard to do it right. If I remember correctly Zope was using some sandboxing and because of it, it took a while time to catch up with newer versions of Python. You had to compile you own Python because the one that came with the Linux distribution was too new. Also If I'm not mistaking I think that PyPy has some sandboxing. Anyway I'll leave this from 2013 here: > After having work during 3 years on a pysandbox project to sandbox untrusted code, I now reached a point where I am convinced that pysandbox is broken by design. Different developers tried to convinced me before that pysandbox design is unsafe, but I had to experience it myself to be convineced. > It would also be nice to help developers looking for a sandbox for their application. Please tell me if you know sandbox projects for Python so I can redirect users of pysandbox to a safer solution. I already know PyPy sandbox. -- https://mail.python.org/pipermail/python-dev/2013-November/130132.html https://mail.python.org/pipermail/python-dev/2013-November/1...

0xblinq 4y ago

Wouldn't running your development environment inside docker provide the same safety levels?

louislang 4y ago
Containers aren't great security boundaries. To get the safety you'd really need, you should absolutely use a VM.

ryukafalz 4y ago

I broadly agree, but if I'm reading your suggestion correctly I think that "access" list is too coarse-grained still! It looks like you're suggesting a predefined list of permissions that can be granted to a dependency, but... why not go even further? If the list of things you're passing in are references rather than strings, then you could do... const apollo = require('apollo-client', {fetch}); const stringutils = require('stringutils'); That way, you can pass in at a very fine-grained level any object you want to, including things like `fetch` and `fs`. But then, you could just as easily pass in a custom `loggedFetch` module like so, that wraps `fetch` and logs all network requests made: const apollo = require('apollo-client', {fetch: loggedFetch}); This is the object-capability security model, and as a sibling commenter pointed out, LavaMoat basically does this for JavaScript dependencies. (This is also basically just lexical scope, by the way, if your language is strict enough - though most unfortunately aren't.)

FractalHQ 4y ago
I love this idea. But what about peer deps? And their dependencies? What if multiple deps share peer deps? Is there a simple way permissions could be resolved in the web of modules? Also- tangential but I really wish people would stop using require and use ESM whenever possible. I cringe when I see `require` just as much as when I see `var` being used in 2022, but I’m not certain there’s never a good reason not to use ESM.
- nirimda 4y ago
  If you want to override the dependency of your dependency, then it's your dependency too - so if you were using the javascript approach up there you might have var bar = require("bar", {baz: loggingBaz}) var foo = require("foo", {bar}) This is true even if you never otherwise use bar. This is certainly how peer dependencies work in nix flakes, for instance, where you can say something like bar = //.... bar.inputs.baz.follows = loggingBaz; foo = //.... foo.inputs.bar.follows = bar; Since nix flakes is a package/dependency manager, it generates a lockfile that you could inspect too see what all the dependencies are, and make sure you got it right (oh, I still have two bar's, I must have forgotten to override some other dep). I suppose any compiler that involves a linking stage would in principle be able to generate some comparable output at the language level. (Apologies for using require and var, but I'm convinced the functional syntax is semantically clearer in this context.)

bernawil 4y ago

A resource access model for dependencies doesn't make much sense to me, there's basically only 2 things you want to gate access for libraries: filesystem and network. And it's all-in. A library that needs network access may be legit today and after an update start exfiltrating data to a different url. It seems easier to grep for fs and network calls in the library code than any of that.

jononor 4y ago
Restricting a process to only be able to access opt-in list of directories underneath the project directory would be useful. Assuming one uses a venv, and all the dependencies are contained there. Then one might want some data folder. And have at least prevented dels from scraping user-wide secrets.
- bernawil 4y ago
  You're describing a chroot jail. key there is "process". Dealing with processes permissions is the OS's job. A if a language wants to deal in library's security it should strive to make static analysis possible. Eg: the language guarantees that network and filesystem calls can only be done with a single function, statically so I can audit that leftpad indeed doesn't make network calls .

mdavidn 4y ago

In a post-Spectre and post-rowhammer world, I have my doubts that any sub-process security boundary will prove durable in the long run.

marmada 4y ago

Wasmtime / WASI does this extremely well.

hayley-patton 4y ago

Newspeak? https://newspeaklanguage.org/ https://newspeaklanguage.org/

pjmlp 4y ago

Java and .NET had it, and in both cases they eventually dropped it, because many security exploits were cause by developers not really understanding how to use them. In the end they became yet another attack vector, and now everyone should use OS security services instead.

lionkor 4y ago

Sort of like BSD's pledge() and similar APIs[1] [1]: https://man.openbsd.org/pledge.2 https://man.openbsd.org/pledge.2