3 ms·

I have become increasingly concerned about non-privilege escalation security breaches. As a developer, I run so much code in the form of tools, libraries, etc w

I have become increasingly concerned about non-privilege escalation security breaches. As a developer, I run so much code in the form of tools, libraries, etc which are barely vetted, yet they have full access to read and upload everything on my system. SSH keys, credentials, env files are all a minor project takeover away because there is effectively zero user space isolation. Running everything inside a guest VM is looking like an unceasingly sane strategy.

jeroenhd 3y ago
I've worked with someone who insisted on running every different project or third party tool in a separate Vagrant machine. At first I scoffed at it, because it seemed like such a waste of resources (this was before VT-d was mainstream let alone other optimizations), but over the years I've only gained more respect for their approach. One problem that I always come back to when I try to set up such restrictions if the lack of GPU access from a VM. A relatively recent standard for virtual GPUs should work on all major players, but both on Intel and Nvidia I've never gotten it to work without crashes or glitches on my hardware. These days, Linux's containerization API seems mature enough to set usable limits. It's possible to escape containers through kernel exploits and misconfigurations (i.e. the "docker-in-docker" setups) but honestly I'm mostly trying to protect one of the million files yarn downloads for me to read my ~/.ssh directory. I believe systemd-run should allow me to run many dev tools with proper sandboxing if I call it right, but I haven't bothered to read up on it yet. I'm also slightly annoyed at the lack of good tooling for setting up quick and dirty network namespaces. The tools I've seen seem to assume that whatever namespace I want to set up will be part of my permanent machine setup, with changes to system daemon config all over the place. Another path to consider is switching to more hardware backed storage. I've put my SSH key in my TPM and no malware is ever going to exfiltrate the private key from there. I've thought up some theoretical solutions to the API access problem and the best I've been able to come up with has been a TPM-backed dom0 VM with the rest of the system in dom1, all other hardware forwarded through VT-d and friends, with access to the dom0 VM only through some kind of interactive protocol, possibly with some kind of network-based permission prompt before key access (i.e. click allow on your phone for the secret VM to share the product key). It would probably work, but I have no illusions about someone making something like that that's going to be usable on a normal user desktop, let alone a mobile device (with all of its power management challenges).
- overbytecode 3y ago
  This is literally all I can think about these days. I’m getting paranoid and I want to start my OS from a clean slate and use sandboxing for every project. I spend so much time auditing random scripts and libraries because I run them on my laptop. But sandboxing options aren’t that great right now. Docker doesn’t cut it because it isn’t a security sandbox, and full blown VMs with vagrant have too much friction and needs lots of resources. I think this is something where Firecracker (Amazon’s lightweight VM) can really shine. But it needs a better a DevEx to act as a disposable/reusable environment that’s easy to start up, easy to specify dependencies. Maybe something uniting firecracker and nix would be the sweet spot for both isolation and reproducibility. I’m rambling but it feels like this should be a solved problem, maybe it is and I don’t know where to look.
jerry1979 3y ago
Why not set up different users to do different tasks?
- 0cf8612b2e1e 3y ago
  That is probably an option, but I would have some practical concerns. I know Linux is multi-user, but given the majority of systems are single-user, I wonder how well modern software deals with that kind of environment. 1) Can I run the project-user account within my real account? Having to launch a DE as the dedicated guest would quickly grow old as I need to multi-task work, and each account would need all sorts of common configuration (browser, email, chat applications, credentials to server X). Which would end up leaking many secrets to the project-account. If I can run the project-account within my real account - do all GUI apps work? Can I launch VS Code as the project-user without issue? Or will extensions make some assumptions about logged in user vs user account running the application and bork things? 2) How good do I have to be in my op-sec? Running a project-account is going to encounter friction where some resource is "local" to my real account but denied to the project-account. What do I have to do to give limited peak to that data? How easy would it be to slip up and make the entire exercise a charade because I screwed up the isolation? Running a VM is probably little different, but I think the isolated-by-default would make the security boundaries more clear and maintainable. Especially since the VS Code Remote Extensions (the best part of the platform) should make this kind of workflow more practical.

jeroenhd 3y ago

I've worked with someone who insisted on running every different project or third party tool in a separate Vagrant machine. At first I scoffed at it, because it seemed like such a waste of resources (this was before VT-d was mainstream let alone other optimizations), but over the years I've only gained more respect for their approach. One problem that I always come back to when I try to set up such restrictions if the lack of GPU access from a VM. A relatively recent standard for virtual GPUs should work on all major players, but both on Intel and Nvidia I've never gotten it to work without crashes or glitches on my hardware. These days, Linux's containerization API seems mature enough to set usable limits. It's possible to escape containers through kernel exploits and misconfigurations (i.e. the "docker-in-docker" setups) but honestly I'm mostly trying to protect one of the million files yarn downloads for me to read my ~/.ssh directory. I believe systemd-run should allow me to run many dev tools with proper sandboxing if I call it right, but I haven't bothered to read up on it yet. I'm also slightly annoyed at the lack of good tooling for setting up quick and dirty network namespaces. The tools I've seen seem to assume that whatever namespace I want to set up will be part of my permanent machine setup, with changes to system daemon config all over the place. Another path to consider is switching to more hardware backed storage. I've put my SSH key in my TPM and no malware is ever going to exfiltrate the private key from there. I've thought up some theoretical solutions to the API access problem and the best I've been able to come up with has been a TPM-backed dom0 VM with the rest of the system in dom1, all other hardware forwarded through VT-d and friends, with access to the dom0 VM only through some kind of interactive protocol, possibly with some kind of network-based permission prompt before key access (i.e. click allow on your phone for the secret VM to share the product key). It would probably work, but I have no illusions about someone making something like that that's going to be usable on a normal user desktop, let alone a mobile device (with all of its power management challenges).

overbytecode 3y ago
This is literally all I can think about these days. I’m getting paranoid and I want to start my OS from a clean slate and use sandboxing for every project. I spend so much time auditing random scripts and libraries because I run them on my laptop. But sandboxing options aren’t that great right now. Docker doesn’t cut it because it isn’t a security sandbox, and full blown VMs with vagrant have too much friction and needs lots of resources. I think this is something where Firecracker (Amazon’s lightweight VM) can really shine. But it needs a better a DevEx to act as a disposable/reusable environment that’s easy to start up, easy to specify dependencies. Maybe something uniting firecracker and nix would be the sweet spot for both isolation and reproducibility. I’m rambling but it feels like this should be a solved problem, maybe it is and I don’t know where to look.

jerry1979 3y ago

Why not set up different users to do different tasks?

0cf8612b2e1e 3y ago
That is probably an option, but I would have some practical concerns. I know Linux is multi-user, but given the majority of systems are single-user, I wonder how well modern software deals with that kind of environment. 1) Can I run the project-user account within my real account? Having to launch a DE as the dedicated guest would quickly grow old as I need to multi-task work, and each account would need all sorts of common configuration (browser, email, chat applications, credentials to server X). Which would end up leaking many secrets to the project-account. If I can run the project-account within my real account - do all GUI apps work? Can I launch VS Code as the project-user without issue? Or will extensions make some assumptions about logged in user vs user account running the application and bork things? 2) How good do I have to be in my op-sec? Running a project-account is going to encounter friction where some resource is "local" to my real account but denied to the project-account. What do I have to do to give limited peak to that data? How easy would it be to slip up and make the entire exercise a charade because I screwed up the isolation? Running a VM is probably little different, but I think the isolated-by-default would make the security boundaries more clear and maintainable. Especially since the VS Code Remote Extensions (the best part of the platform) should make this kind of workflow more practical.