4 ms·

Curious to know why existing sandboxing/isolation technologies like VMs & Containers not enough here?

simonw 3y ago
They're not lightweight and easy enough to use. I have Docker on my Mac. I run things in it if I have to, but it's pretty resource heavy and I have to look at my notes every time I want to run "docker run". Plus Docker isn't actually a fully secure sandbox! There's Firecracker, which I believe is secure (AWS built it for Lambda after all) but good luck figuring out how to run that on a Mac. I want something that's tiny, fast, ridiculously easy to use and that I can run dozens if not hundreds of instances. WebAssembly is almost that thing.
- rrherr 3y ago
  Agreed about Docker. Personally, I use Google Colab for my convenient, lightweight Python sandbox. But I’d rather not depend on Google. It feels weird that this is my best option for now.
  - Intralexical 3y ago
    > Personally, I use Google Colab for my convenient, lightweight Python sandbox. But I’d rather not depend on Google. It feels weird that this is my best option for now. `bwrap` is great for this on Linux. A one line alias should let you run any program in a lightweight and highly customizable sandbox safely-ish. See my reply to the parent comment for an example.
- Intralexical 3y ago
  > They're not lightweight and easy enough to use. ``` $ alias SafeRun="bwrap --ro-bind / / --dev /dev --proc /proc --unshare-all --tmpfs ~ --bind ~/Untrusted ~/Untrusted" $ SafeRun python3 ~/Untrusted/RandomScript.py $ SafeRun ~/Untrusted/RandomExecutable ``` This is basically manually invoking what Flatpak does: https://github.com/containers/bubblewrap https://github.com/containers/bubblewrap This is also useful for more than just security. E.G., you can test how your app would behave on a fresh install by masking your user configuration files. I personally also have a tool that uses it to basically bundle all dependencies from an entire Linux distribution in order to make highly portable AppImages— Been meaning to post that, will get around to it eventually maybe. The flags above should hide your user data (`--tmpfs`), disable network access (`--unshare-all`), hide/virtualize devices and OS state (`--dev` and `--proc`), and make the rest of the root filesystem read-only (`--ro-bind`— Including the insecure X11 socket in `/tmp`, which you might want to expose for GUI apps). Check them against `bwrap --help`; I might have omitted one or two more things you'd need.
- ashishbijlani 3y ago
  > There's Firecracker, which I believe is secure (AWS built it for Lambda after all) but good luck figuring out how to run that on a Mac. Then, how about using AWS Lambda for your use case?
  - simonw 3y ago
    I want to run code on my own laptop and phone.
    - ashishbijlani 3y ago
      Makes sense. When I read your notes, it appeared that you wanted to run untrusted code on a backend server.
      - simonw 3y ago
        Yeah, I want to do that as well! And Lambda is definitely high on my list of potential solutions for that. I'm still hoping for a solution to the run-on-personal-devices problem though.
- g_p 3y ago
  Interestingly, Docker on Mac actually runs lightweight VM, which then runs a whole Linux OS inside it, to then run your containers. From an isolation perspective, that's giving you hypervisor level isolation which is a step above most container-based solutions, which you point out rightly isn't a full sandbox. A hypervisor with CPU support is probably the gold standard for isolation, asides from running it on a separate physically isolated system.
internet101010 3y ago
It's really just a matter of practicality. Developing within docker or a VM is kind of a pain.
- ashishbijlani 3y ago
  Would love to know the specifics I'm missing. Based on my experience, Docker/FC just works out of the box and gives you a shell.
  - dagw 3y ago
    A big one that has bitten my team a lot is syncing files between local and container. There are even whole separate tools like https://docker-sync.readthedocs.io https://docker-sync.readthedocs.io designed to try to mitigate the problem. The most annoying thing is it almost works almost all the time and then all of a sudden stop working for some people for no obvious reason. We gave up on Docker for development because we couldn't trust it to work consistently.
    - taway1237 3y ago
      I must be missing something, but why not just mount your code into the container? That's what I usually do.
      - dagw 3y ago
        I have a directory with my files in it and mount that directory into the container. However when I make changes to the files outside the container, those changes aren't always reflected inside the container, causing all kinds of weird behaviour and mismatches. It almost always works as expected, for almost everybody on the team, but everybody with a Mac has experienced this at least once.

They're not lightweight and easy enough to use. I have Docker on my Mac. I run things in it if I have to, but it's pretty resource heavy and I have to look at my notes every time I want to run "docker run". Plus Docker isn't actually a fully secure sandbox! There's Firecracker, which I believe is secure (AWS built it for Lambda after all) but good luck figuring out how to run that on a Mac. I want something that's tiny, fast, ridiculously easy to use and that I can run dozens if not hundreds of instances. WebAssembly is almost that thing.

rrherr 3y ago
Agreed about Docker. Personally, I use Google Colab for my convenient, lightweight Python sandbox. But I’d rather not depend on Google. It feels weird that this is my best option for now.
- Intralexical 3y ago
  > Personally, I use Google Colab for my convenient, lightweight Python sandbox. But I’d rather not depend on Google. It feels weird that this is my best option for now. `bwrap` is great for this on Linux. A one line alias should let you run any program in a lightweight and highly customizable sandbox safely-ish. See my reply to the parent comment for an example.
Intralexical 3y ago
> They're not lightweight and easy enough to use. ``` $ alias SafeRun="bwrap --ro-bind / / --dev /dev --proc /proc --unshare-all --tmpfs ~ --bind ~/Untrusted ~/Untrusted" $ SafeRun python3 ~/Untrusted/RandomScript.py $ SafeRun ~/Untrusted/RandomExecutable ``` This is basically manually invoking what Flatpak does: https://github.com/containers/bubblewrap https://github.com/containers/bubblewrap This is also useful for more than just security. E.G., you can test how your app would behave on a fresh install by masking your user configuration files. I personally also have a tool that uses it to basically bundle all dependencies from an entire Linux distribution in order to make highly portable AppImages— Been meaning to post that, will get around to it eventually maybe. The flags above should hide your user data (`--tmpfs`), disable network access (`--unshare-all`), hide/virtualize devices and OS state (`--dev` and `--proc`), and make the rest of the root filesystem read-only (`--ro-bind`— Including the insecure X11 socket in `/tmp`, which you might want to expose for GUI apps). Check them against `bwrap --help`; I might have omitted one or two more things you'd need.
ashishbijlani 3y ago
> There's Firecracker, which I believe is secure (AWS built it for Lambda after all) but good luck figuring out how to run that on a Mac. Then, how about using AWS Lambda for your use case?
- simonw 3y ago
  I want to run code on my own laptop and phone.
  - ashishbijlani 3y ago
    Makes sense. When I read your notes, it appeared that you wanted to run untrusted code on a backend server.
    - simonw 3y ago
      Yeah, I want to do that as well! And Lambda is definitely high on my list of potential solutions for that. I'm still hoping for a solution to the run-on-personal-devices problem though.
g_p 3y ago
Interestingly, Docker on Mac actually runs lightweight VM, which then runs a whole Linux OS inside it, to then run your containers. From an isolation perspective, that's giving you hypervisor level isolation which is a step above most container-based solutions, which you point out rightly isn't a full sandbox. A hypervisor with CPU support is probably the gold standard for isolation, asides from running it on a separate physically isolated system.

internet101010 3y ago

It's really just a matter of practicality. Developing within docker or a VM is kind of a pain.

ashishbijlani 3y ago
Would love to know the specifics I'm missing. Based on my experience, Docker/FC just works out of the box and gives you a shell.
- dagw 3y ago
  A big one that has bitten my team a lot is syncing files between local and container. There are even whole separate tools like https://docker-sync.readthedocs.io https://docker-sync.readthedocs.io designed to try to mitigate the problem. The most annoying thing is it almost works almost all the time and then all of a sudden stop working for some people for no obvious reason. We gave up on Docker for development because we couldn't trust it to work consistently.
  - taway1237 3y ago
    I must be missing something, but why not just mount your code into the container? That's what I usually do.
    - dagw 3y ago
      I have a directory with my files in it and mount that directory into the container. However when I make changes to the files outside the container, those changes aren't always reflected inside the container, causing all kinds of weird behaviour and mismatches. It almost always works as expected, for almost everybody on the team, but everybody with a Mac has experienced this at least once.