13 ms·

Microsandbox: Virtual Machines that feel and perform like containers

402 points by makeboss 1y ago

deleted 1y ago
[deleted]
appcypher 1y ago
Thanks for sharing! I'm the creator of microsandbox. If there is anything you need to know about the project, let me know. This project is meant to make creating microvms from your machine as easy as using Docker containers. Ask me anything.
- esafak 1y ago
  Looks neat. If I understand correctly, I can use it to spin up backends on the fly? You have an ambitious list of languages to support: https://github.com/microsandbox/microsandbox/tree/main/sdk https://github.com/microsandbox/microsandbox/tree/main/sdk edit: A fleshed out contributors guide to add support for a new language would help. https://github.com/microsandbox/microsandbox/blob/main/CONTRIBUTING.md https://github.com/microsandbox/microsandbox/blob/main/CONTR...
  - appcypher 1y ago
    Yes. Self-hosting and using it on your own backend infra is the main use-case. And JVM support should just work since it is a Linux machine.
- 0cf8612b2e1e 1y ago
  Only did a quick skim of the readme, but a few questions which I would like some elaboration. How is it so fast? Is it making any trade offs vs a traditional VM? Is there potential the VM isolation is compromised? Can I run a GUI inside of it? Do you think of this as a new Vagrant? How do I get data in/out?
  - appcypher 1y ago
    > How is it so fast? Is it making any trade offs vs a traditional VM? Is there potential the VM isolation is compromised? It is a lighweight VM and uses the same technology as Firecracker > Can I run a GUI inside of it? It is planned but not yet implemented. But it is absolutely possible. > Do you think of this as a new Vagrant? I would consider Docker for VMs instead. In a similar way, it focuses on dev ops type use case like deplying apps, etc. > How do I get data in/out? There is an SDK and server that help does that and file streaming is planned. But right now, you can execute commands in the VM and get the result back via the server
    - westurner 1y ago
      > I would consider Docker for VMs instead. Native Containers would probably solve here, too. From https://news.ycombinator.com/item?id=43553198 https://news.ycombinator.com/item?id=43553198 : >>> ostree native containers are bootable host images that can also be built and signed with a SLSA provenance attestation; https://coreos.github.io/rpm-ostree/container/ https://coreos.github.io/rpm-ostree/container/ And also from that thread: > How should a microkernel run (WASI) WASM runtimes? What is the most minimal microvm for WASM / WASI, and what are the advantages to running WASM workloads with firecracker or microsandbox?
      - appcypher 1y ago
        > What is the most minimal microvm for WASM / WASI, By setting up an image with wasmtime for example. > and what are the advantages to running WASM workloads with firecracker or microsandbox? I can think of stronger isolation or when you have legacy stuff you need to run alongside.
        westurner 1y ago
        From https://e2b.dev/blog/firecracker-vs-qemu https://e2b.dev/blog/firecracker-vs-qemu > AWS built [Firecracker (which is built on KVM)] to power Lambda and Fargate [2], where they need to quickly spin up isolated environments for running customer code. Companies like E2B use Firecracker to run AI generated code securily in the cloud, while Fly.io uses it to run lightweight container-like VMs at the edge [4, 5]. "We replaced Firecracker with QEMU" (2023) https://news.ycombinator.com/item?id=36666782 https://news.ycombinator.com/item?id=36666782 "Firecracker's Kernel Support Policy" describes compatible kernel configurations; https://github.com/firecracker-microvm/firecracker/blob/main/docs/kernel-policy.md https://github.com/firecracker-microvm/firecracker/blob/main... /? wasi microvm kernel [github] https://www.google.com/search?q=wasi+microvm+kernel+GitHub https://www.google.com/search?q=wasi+microvm+kernel+GitHub : - "Mewz: Lightweight Execution Environment for WebAssembly with High Isolation and Portability using Unikernels" (2024) https://arxiv.org/abs/2411.01129 https://arxiv.org/abs/2411.01129 similar: https://scholar.google.com/scholar?q=related:b3657VNcyJ0J:scholar.google.com/&scioq=Mewz:+Lightweight+Execution+Environment+for+WebAssembly+with+High+Isolation+and+Portability+using+Unikernels&hl=en&as_sdt=0,43 https://scholar.google.com/scholar?q=related:b3657VNcyJ0J:sc...
- hugs 1y ago
  Looks great! This might be extremely useful for a distributed/decentralized software testing network I'm building (called Valet Network)... Question: How does networking work? Can I restrict/limit microvms so that they can only access public IP addresses? (or in other words... making sure the microvms can't access any local network IP addresses)
  - appcypher 1y ago
    Yes! With the `scope` property. https://github.com/microsandbox/microsandbox/blob/0c13fc27ab687a28f56143bd3ce66769e840eb62/microsandbox-core/lib/config/microsandbox/config.rs#L222 https://github.com/microsandbox/microsandbox/blob/0c13fc27ab...
    - hugs 1y ago
      thanks! have an example on how to use that in a sandboxfile? (also, this project is really cool. great work!)
      - appcypher 1y ago
        Yeah. I need to fix that in the docs!
        hugs 1y ago
        no prob!
- simonw 1y ago
  What's the story for macOS support?
  - deleted 1y ago
    [deleted]
  - appcypher 1y ago
    It uses libkrun which uses Hypervisor.framework on macOS.
- wolfhumble 1y ago
  Can you use Microsandbox for everything you can use Docker for, or are there cases where containers make more sense? Congratulations on the launch!
  - appcypher 1y ago
    We want microsandbox to be usable for everything you can with Docker. That said, hosting microVMs require dedicated hardware or VMs with nested virt support. Containers don’t have that problem.
- nqzero 1y ago
  i'm on a mid-level laptop, at times with slow or expensive internet, running ubuntu. i want to be able to run nominally-isolated "copies" of my laptop at near-native speed 1. each one should have it's own network config, eg so i can use wireguard or a vpn 2. gui pass-through to the host, eg wayland, for trusted tools, eg firefox, zoom or citrix 3. needs to be lightweight. eg gnome-boxes is dead simple to setup and run and it works, but the resource usage was noticeably higher than native 4. optional - more security is better (ie, i might run semi-untrusted software in one of them, eg from a github repo or npm), but i'm not expecting miracles and accept that escape is possible 5. optional - sharing disk with the host via COW would be nice, so i'd only need to install the env-specific packages, not the full OS i'm currently working on a podman solution, and i believe that it will work (but rebuilding seems to hammer the network - i'm hoping i can tweak the layers to reduce this). does microsandbox offer any advantages for this use case ?
  - appcypher 1y ago
    > 1. each one should have it's own network config, eg so i can use wireguard or a vpn This is possible right now but the networking is not where I want it to be yet. It uses libkrun's default TSI impl; performant and simplifies setup but can be inflexible. I plan to implement an alternative user-space networking stack soon. > 2. gui pass-through to the host, eg wayland, for trusted tools, eg firefox, zoom or citrix We don't have GUI passthrough. VNC? > 3. needs to be lightweight. eg gnome-boxes is dead simple to setup and run and it works, but the resource usage was noticeably higher than native It is lightweight in the sense that it is not a full vm > 4. optional - more security is better (ie, i might run semi-untrusted software in one of them, eg from a github repo or npm), but i'm not expecting miracles and accept that escape is possible The security guarantees are similar to what typical VMs support. It is hardware-virtualized so I would say you should be fine. > 5. optional - sharing disk with the host via COW would be nice, so i'd only need to install the env-specific packages, not the full OS Yeah. It uses virtio-fs and has overlayfs on top of that for COW.
- simonw 1y ago
  I'm trying this out now and it's very promising. One problem I'm running into with the Python library is that I'd like to keep that sandbox running for several minutes while I do things like set variables in one call and then use them for stuff several calls later. I keep seeing this error intermittently: Error: Sandbox is not started. Call start() first Is there a suggested way of keeping a sandbox around for longer? The documented code pattern is this: async def main(): async with PythonSandbox.create(name="my-sandbox") as sb: exec = await sb.run("print('Hello, World!')") print(await exec.output()) Due to the way my code works I want to instantiate the sandbox once for a specific class and then have multiple calls to it by class methods, which isn't a clean fit for that "async with" pattern. Any recommendations?
  - appcypher 1y ago
    Right. You can skip the `with` context manager and call start and stop yourself. There is an example of that here: https://github.com/microsandbox/microsandbox/blob/0c13fc27ab687a28f56143bd3ce66769e840eb62/sdk/python/examples/command.py#L129-L153 https://github.com/microsandbox/microsandbox/blob/0c13fc27ab...
  - gcharbonnier 1y ago
    async with is just syntactic sugar. You could very well call __aenter__ and __aexit__ manually. You could also use an AsyncExitStack, call __aenter__ manually, then enter_async_context, and call aclose when you’re done. Since aclose method exists I guess this is not an anti-pattern. https://docs.python.org/3/library/contextlib.html#contextlib.AsyncExitStack https://docs.python.org/3/library/contextlib.html#contextlib...
- codethief 1y ago
  Hi appcypher, very cool project! Does the underlying MicroVM feature provide an OCI runtime interface, so that it could be used as a replacement for runc/crun in Docker/Podman?
  - Nypro 1y ago
    No. Not yet. Would be nice to have
    - codethief 1y ago
      Thanks for your response! One more question: What syscalls do I need to have access to in order to run a MicroVM? I'm asking because ideally I'd like to run container workloads inside existing containers (self-hosted GitLab CI runners) whose configuration (including AppArmor) I don't control.
- Hilift 1y ago
  Are you ready for the deluge of networking questions for all the buck wild configurations?
  - Nypro 1y ago
    Lol. I should brace for impact. Networking continues to be a pain but I'm open to suggestions.
- catlifeonmars 1y ago
  How does the microvm architecture compare with firecracker?
  - appcypher 1y ago
    They are similar. We use libkrun under the hood. Firecracker team seems not to be interested in a macOS implementation
    - catlifeonmars 1y ago
      Ah gotcha! I was unaware that firecracker used KVM under the hood.
- nulld3v 1y ago
  Cool project. Off topic question: Are the images in the "Use Cases" section in the README from a real app? I like the clean UI design.
  - appcypher 1y ago
    No they are not.
- spicybright 1y ago
  I like the idea. But when you say "bullet proof" security, there are exploits to break out of VMs that exist. Have you looked into those?
  - appcypher 1y ago
    Will fix the docs
- meander_water 1y ago
  Can you explain how this compares to Kata Containers? [0] That also supports OCI to run microVMs. You can also choose different hypervisors such as firecracker to run it on. [0] https://katacontainers.io/ https://katacontainers.io/
  - appcypher 1y ago
    Katacontainers is an interesting project. Microsandbox is a more opinionated project with a UX that focuses on getting up and running with microVMs quickly. I want this experience for Linux, macOS and Windows users. More importantly is making sandboxing really accessible to AI devs with `msb server`.
- nikolamus 1y ago
  Think I can build a notebook on top of this ? Jupyter client has been a pain to manage
  - appcypher 1y ago
    Not sure what that entails. You can try and I can help along the way
- int_19h 1y ago
  This is very neat tech, but I think you might want to wait until you actually have Windows covered before making claims like https://github.com/microsandbox/microsandbox/blob/main/MSB_V_DOCKER.md#2-true-cross-platform-consistency https://github.com/microsandbox/microsandbox/blob/main/MSB_V...
  - appcypher 1y ago
    What do you mean?
Tsarp 1y ago
Wow. This looks awesome. Can we build our own python sandbox using the sandboxfile spec? This is if I want to add my own packages. Would this be just having my own requirements file here - https://github.com/microsandbox/microsandbox/blob/main/MSB_V_DOCKER.md#3-unified-configuration-model-the-sandboxfile https://github.com/microsandbox/microsandbox/blob/main/MSB_V...
- appcypher 1y ago
  Thank you! > Can we build our own python sandbox using the sandboxfile spec? Yes and I plan to make that work with the SDK. PS: Multi-stage build is WIP.
  - Tsarp 1y ago
    Great will join the discord. Is this embeddable? Will it work with a cross platform desktop app(Tauri)?
    - deleted 1y ago
      [deleted]
    - apitman 1y ago
      An embeddable library that lets you launch Linux VMs that works across Windows, MacOS, and Linux hosts would be incredible.
    - appcypher 1y ago
      If by embeddable, you mean having the vm run in the same process, then no. The vm aborts its process when it's done so it has to run as separate process.
deleted 1y ago
[deleted]
jauntywundrkind 1y ago
Why not some of the existing microvm efforts? Cloud Hypervisor and Firecracker both have an excellent reputation for ultra lightweight VM's. Both are usable in the very popular Kata Containers project (as well as other upstart VM's Dragonball, & StratoVirt). In us by for example the CNCF Confidential Containers https://github.com/kata-containers/kata-containers/blob/main/docs/hypervisors.md#types https://github.com/kata-containers/kata-containers/blob/main... https://confidentialcontainers.org/ https://confidentialcontainers.org/ There's also smaller efforts such as firecracker-containerd or Virtink, both which bring OCI powered microvms into a Docker like position (easy to slot into Kubernetes), via Firecracker and Cloud Hypervisor respectively. https://github.com/smartxworks/virtink https://github.com/smartxworks/virtink https://github.com/firecracker-microvm/firecracker-containerd https://github.com/firecracker-microvm/firecracker-container... Poking around under the hood, microsandbox appears to use krun. There is krunvm for OCI support (includes MacOS/arm64 support!). https://github.com/containers/krunvm https://github.com/containers/krunvm https://github.com/slp/krun https://github.com/slp/krun The orientation as a safe sandbox for AI / MCP tools is a very nicely packaged looking experience, and very well marketred. Congratulations! I'm still not sure why this warrants being it's own project.
- simonw 1y ago
  If we get enough of these sandboxes, maybe we will finally get one that's easy for me to run on my own machines.
  - mike_hearn 1y ago
    Which platforms do you use?
    - simonw 1y ago
      macOS on my laptop, anything that runs in a container for when I deploy things.
      - tough 1y ago
dataflow 1y ago
Tangential question: why does it normally take so long to start traditional VMs in the first place? At least on Windows, if you start a traditional VM, it takes several seconds for it to start running anything. Edit: when I say anything, I'm not talking user programs. I mean as in, before even the first instruction of the firmware -- before even the virtual disk file is zeroed out, in cases where it needs to be. You literally can't pause the VM during this interval because the window hasn't even popped up yet, and even when it has, you still can't for a while because it literally hasn't started running anything. So the kernel and even firmware initialization slowness are entirely irrelevant to my question. Why is that?
- diggan 1y ago
  I mean it is basically booting a computer from scratch, kind of makes sense. You have to allocate memory, start virtual CPUs, initialize devices, run BIOS/UEFI checks, perform hardware enumeration, all that jazz while emulating all of it, which tends to be slower than "real" implementations. I guess there is a bunch of processes for security as well, like wiping like zeroing pages and similar things that takes additional time. If I let a VM use most of my hardware, it takes a few seconds from start to login prompt, which is the same time it takes for my Arch desktop to boot from pressing the button to seeing the login prompt.
  - dataflow 1y ago
    > You have to allocate memory, start virtual CPUs, initialize devices, run BIOS/UEFI checks, perform hardware enumeration, all that jazz while emulating all of it, which tends to be slower than "real" implementations. That's not what I'm asking. I'm saying it takes a long time for it to even execute a single instruction, in the BIOS itself. Even for the window to pop up, before you can even pause the VM (because it hasn't even started yet). What you're describing comes after all that, which I already understand and am not asking about.
    - drewg123 1y ago
      Without any context in terms of what the VM is doing or what VMM software you use, my best guess is that the OS/VMM are pre-allocating memory for the VM. This might involve paging out other processes' memory, which could take some time. I think task manager would tell you if there is a blip of memory usage and paging activity at the time. And I'm sure windows itself has profilers that can tell you what is happening when the VM is started..
Jayakumark 1y ago
Windows support ? and can we VNC in to the sandbox and stream it ?
- appcypher 1y ago
  Windows support is a work in progress. I haven't tested using VNC yet but it should be possible.
- spixy 1y ago
  Windows already has built-in sandbox and its good. https://learn.microsoft.com/en-us/windows/security/application-security/application-isolation/windows-sandbox/ https://learn.microsoft.com/en-us/windows/security/applicati...
h1fra 1y ago
Can't wait to test, if it's really what's advertised it would be much easier to use than workerd or firecracker
McAlpine5892 1y ago
This looks awesome. The amount of super lightweight and almost-disposable VM options in recent years is crazy. I remember when VMs were slow, clunky, and generally painful. I wonder how this compares to Orbstack's [0] tech stack on macOS, specifically the "Linux machines" [1] feature. Seems like Orb might reuse a single VM? --- [0] https://orbstack.dev https://orbstack.dev [1] https://docs.orbstack.dev/machines/ https://docs.orbstack.dev/machines/
jbverschoor 1y ago
Related, https://github.com/jrz/container-shell https://github.com/jrz/container-shell which uses docker to create adhoc shells / chroots in the current directory.
manveru 1y ago
Are the SDKs AI generated? I looked at the Crystal, Ruby, and Zig ones and all they contain is a hello world example with some docs that have little to do with the code. Sorry if this comment seems rude, just curious.
- appcypher 1y ago
  The other SDKs are generated hello-worlds at the moment. I will get to them one by one, but I welcome and appreciate any contributions to them.
jmehman 1y ago
I've been looking for something I could host for this kind of thing - for LLM agents. Ended up on https://www.daytona.io/ https://www.daytona.io/ as I couldn't find anything suitable to self host and realised it was a complex thing to manage. It seems Daytona is open source, including the server platform, but there is no documentation for the server element. Azure also seem to offer a service for this, it's a space that is growing rapidly.
- appcypher 1y ago
  Microsandbox is for people that would like to maintain their own infra. I'm not going to stop trying to make it better to self-host.
  - jmehman 1y ago
    Yeah, it looks great, makes me reconsider the self hosted route
patrick4urcloud 1y ago
very nice ! i will definetly try
ATechGuy 1y ago
Congrats on launching! Booting VMs in milliseconds is certainly important, but it can also be achieved with CloudHypervisor/Firecracker. Where Containers beat VMs is runtime perf. The overhead in case of VMs stems from emulation of IO devices. I believe the overhead will become noticeable for AI agentic use cases. Any plans to address perf issues?
- appcypher 1y ago
  You are right. We leverage libkrun. Libkrun uses virtio-mmio transport for block, vsock and virtio-fs to keep overhead minimal so we basically depend on any perf improvement made upstream. Firecracker is no different btw and E2B uses that for agentic AI workloads. Anyway, I don't have any major plan except fix some issues with the filesystem rn.
SwiftyBug 1y ago
Kind of almost off-topic: I'm working on a project where I must run possibly untrusted JavaScript code. I want to run it in an isolated environment. This looks like a very nice solution as I could spin up a microsandbox and securely run the code. I could even have a pool os live sandboxes so I wouldn't even experience the 200ms starts. Because this is OCI-compatible, I could even provide a whole sandboxed environment on which to run that code. Would that be a good use case for this? Are there better alternatives?
- appcypher 1y ago
  > Would that be a good use case for this? That is an ideal use case > Are there better alternatives? Created microsandbox because I didn't find any
  - SwiftyBug 1y ago
    Awesome. This is really good timing. I'm going to give it a try.
  - spixy 1y ago
    Windows has built-in sandbox and its good. https://learn.microsoft.com/en-us/windows/security/application-security/application-isolation/windows-sandbox/ https://learn.microsoft.com/en-us/windows/security/applicati...
- ericb 1y ago
  runsc / gVisor is interesting also as the runsc engine can be run from within Docker/Docker Desktop. gVisor has performance problems, though. Their data shows 1/3rd the throughput vs. docker runtime for concurrent network calls--if that's an issue for your use-case.
- apitman 1y ago
  You might be able to get away with running QuickJS compiled to WebAssembly: https://til.simonwillison.net/npm/self-hosted-quickjs https://til.simonwillison.net/npm/self-hosted-quickjs
deleted 1y ago
[deleted]
hinkley 1y ago
How’s performance? What’s the overhead versus docker? Terraform or Pulumi integration on the horizon?
- appcypher 1y ago
  Wow. Just seeing this. I've not done proper benchmarking yet but rn we are lagging behind in file I/O for the OverlayFS impl
  - hinkley 1y ago
    There was a period where NFS was faster, particularly on windows and OSX where you were paying a double indirection. Overlays are always tough because docker doesn’t like you writing to the filesystem in the first place. The weapon if first result is deflection; tell them not to do it. I had to put up with an old docker version that leaked overlay data for quite a while before we moved off prem.
elwebmaster 1y ago
One topic I am not finding anything about is networking. Can these microsandbox instances listen on ports? How is the port forwarding configured? Can they access the internet or any resources on the host?
- appcypher 1y ago
  They can. I need to improve the doc. Working on that right now
zackmorris 1y ago
This is great! I'd like to see a formal container security grade that works like: 1) Curate a list of all known (container) exploits 2) Run each exploit in environments of increasing security like permissions-based, jail, Docker and emulator 3) The percentage of prevented exploits would be the score from 0-100% Under this scheme, I'd expect naive attempts at containerization with permissions and jails to score around 0%, while Docker might be above 50% and Microsandbox could potentially reach 100%. This might satisfy some of our intuition around questions like "why not just use a jail?". Also the containers could run on a site on the open web as honeypots with cash or crypto prizes for pwning them to "prove" which containers achieve 100%. We might also need to redefine what "secure" means, since exploits like Rowhammer and Spectre may make nearly all conventional and cloud computing insecure. Or maybe it's a moving target, like how 64 bit encryption might have once been considered secure but now we need 128 bit or higher. Edit: the motivation behind this would be to find a container that's 100% secure without emulation, for performance and cost-savings benefits, as well as gaining insights into how to secure operating systems by containerizing their various services.
- bjackman 1y ago
  You cannot build a secure container runtime (against malicious containers) because underlying it is the Linux kernel. The only way to make Linux containers a meaningful sandbox is to drastically restrict the syscall API surface available to the sandboxee, which quickly reduces its value. It's no longer a "generic platform that you can throw any workload onto" but instead a bespoke thing that needs to be tuned and reconfigured for every usecase. This is why you need virtualization. Until we have a properly hardened and memory safe OS, it's the only way. And if we do build such an OS it's unclear to me whether it will be faster than running MicroVMs on a Linux host.
  - Veserv 1y ago
    You cannot build a secure virtualization runtime because underlying it is the VMM. Until you have a secure VMM you are subject to precisely the same class of problems plaguing container runtimes. The only meaningful difference is that Linux containers target partitioning Linux kernel services which is a shared-by-default/default-allow environment that was never designed for and has never achieved meaningful security. The number of vulnerabilities resulting from, "whoopsie, we forgot to partition shared service 123" would be hilarious if it were not a complete lapse of security engineering in a product people are convinced is adequate for security-critical applications. Present a vulnerability assessment demonstrating a team of 10 with 3 years time (~10-30 M$, comparable to many commercially-motivated single-victim attacks these days) can find no vulnerabilities in your deployment or a formal proof of security and correctness otherwise we should stick with the default assumption that software if easily hacked instead of the extraordinary claim that demands extraordinary evidence.
rbitar 1y ago
Looks great and excited to try this out. We’ve also had success using CodeSandbox SDK and E2B, can you share some thoughts on how you compare or future direction? Do you also use Firecracker under the hood?
- pkkkzip 1y ago
  I can't tell if it uses firecracker but thats my main question too. I'm curious as to whether microsandbox will be maintained and proper auditing will be done. I welcome alternatives. It's been tough wrestling with Firecracker and OCI images. Kata container is also tough.
  - appcypher 1y ago
    It will be maintained as I will be using it for some other product. And it will be audited in the future but it still early days.
  - pdimitar 1y ago
    I wanted to try Kata containers soon. What difficulties do you have with them?
- appcypher 1y ago
  > can you share some thoughts on how you compare or future direction? Microsandbox does not offer a cloud solution. It is self-hosted, designed to do what E2B does, to make it easier working with microVM-based sandboxes on your local machine whether that is Linux, macOS or Windows (planned) and to seamlessly transition to prod. > Do you also use Firecracker under the hood? It uses libkrun.
  - rbitar 1y ago
    Self-hosting is definitely something we are keen to explore as most of the cloud solutions have resource constrains (ie, total active MicroVMs and/or specs per VM) and managing billing gets complicated even with hibernation features. Great project and we'll definitely take it for a spin
sureglymop 1y ago
Always interested when things like this come up. What like about containers is how quickly I can run something, e.g. `docker run --rm ...` without having to specify disk size, amount of cpu cores, etc. I can then diff the state of the container with the image (and other things) to see what some program did while it ran. So I basically want the same but instead with small vms to have better sandboxing. Sometimes I also use bwrap but it's not really intended to be used on the command line like that.
- srmatto 1y ago
  It has a YAML config format to declare all of that so you could just do that once, or template it, generate it on the fly, fetch it from remote, or many other methods.
eamann 1y ago
> Ever needed to run code you don't fully trust? Then the installation instructions include piping a remote script directly to Bash ... Oh irony ... That said, the concept itself is intriguing.
- deleted 1y ago
  [deleted]
- appcypher 1y ago
  Your statement initially went over my head. Sorry lol. You can always download the installer script and audit yourself. I will set up proper distribution later.
  - hakcermani 1y ago
    .. did exactly that and also changed the BINDIR and LIBDIR to another location. BTW, amazing project from initial glance. Will give it a detailed look this weekend!
  - raphinou 1y ago
    In case you're interested when you set up proper distribution, I'm working on an open source solution aiming to improve security of downloads from the internet. Our first step is maintaining a mirror of checksums published in GitHub releases at https://github.com/asfaload/checksums/ https://github.com/asfaload/checksums/. If you publish a checksums file in your releases it can automatically be mirrored. The checksums mirror is not our end game, but it already protects against changes of released files from the time the mirror was taken. For anyone interested: https://asfaload.com/asfald/ https://asfaload.com/asfald/
amelius 1y ago
For my taste, container technology is pushing the OS too far. By typing: mount you immediately see what I mean. Stuff that should be hidden is now in plain sight, and destroys the usefulness of simple system commands. And worse, the user can fiddle with the data structures. It's like giving the user peek and poke commands. The idea of containers is nice, but they are a hack until kernels are re-architected.
- throwaway314155 1y ago
  Sorry I am lacking the context to understand this post. What does running mount inside a container do that's so egregious? Are host mounts exposed to the container somehow? I thought everything needed to be explicitly passed through to the container (e.g. using a volume)?
  - remram 1y ago
    I think they mean that running `mount` on the host now lists hundreds of mountpoints from containers, snaps, packagekit etc.
- topspin 1y ago
  On recent Linux, try: findmnt --real It's part of linux-utils, so it is generally available wherever have a shell. The legacy tools you have in mind aren't ever going to be changed as you would wish, for reasons.
sbassi 1y ago
There are python and node environment for this, so they are not VMs in the sense that I can host a OS and arbitrary executables?
- appcypher 1y ago
  They are Linux VMs and you can host any executable that can work on that. The python/node environment you see is part of what makes the SDK work. Really, it's very similar to Docker in use.
  - sbassi 1y ago
    thank you. Is there any "docker host" or centralized repo where I can pull VMs from?
    - appcypher 1y ago
      We support just Docker hub for now. Let me know if you want any other OCI-compatible registry. PS: microsandbox will likely have its own OCI registry in the future
airocker 1y ago
Would love to hear nix people take on this?
- mjrusso 1y ago
  As a Nix user, I'm actually really excited to try this out. I want to run sandboxes based on Docker images that have Nix pre-installed. (Once the VM boots, apply the project-specific Flake, and then run Docker Compose for databases and other supporting services.) In theory, an easy-to-use, fully isolated dev environment that matches how I normally develop, except inside of a VM.
  - airocker 1y ago
    but dont they have overlapping requirements of solving "not works on my machine"
    - mjrusso 1y ago
      Microsandbox's primary goal is to make it easy to build environments for running untrusted code. Nix, on the other hand, solves the problem of building reproducible environments... but making said environments safe for running untrusted code is left as an exercise for the reader.

402 points by makeboss 1y ago

deleted 1y ago

[deleted]

Thanks for sharing! I'm the creator of microsandbox. If there is anything you need to know about the project, let me know. This project is meant to make creating microvms from your machine as easy as using Docker containers. Ask me anything.

esafak 1y ago
Looks neat. If I understand correctly, I can use it to spin up backends on the fly? You have an ambitious list of languages to support: https://github.com/microsandbox/microsandbox/tree/main/sdk https://github.com/microsandbox/microsandbox/tree/main/sdk edit: A fleshed out contributors guide to add support for a new language would help. https://github.com/microsandbox/microsandbox/blob/main/CONTRIBUTING.md https://github.com/microsandbox/microsandbox/blob/main/CONTR...
- appcypher 1y ago
  Yes. Self-hosting and using it on your own backend infra is the main use-case. And JVM support should just work since it is a Linux machine.
0cf8612b2e1e 1y ago
Only did a quick skim of the readme, but a few questions which I would like some elaboration. How is it so fast? Is it making any trade offs vs a traditional VM? Is there potential the VM isolation is compromised? Can I run a GUI inside of it? Do you think of this as a new Vagrant? How do I get data in/out?
- appcypher 1y ago
  > How is it so fast? Is it making any trade offs vs a traditional VM? Is there potential the VM isolation is compromised? It is a lighweight VM and uses the same technology as Firecracker > Can I run a GUI inside of it? It is planned but not yet implemented. But it is absolutely possible. > Do you think of this as a new Vagrant? I would consider Docker for VMs instead. In a similar way, it focuses on dev ops type use case like deplying apps, etc. > How do I get data in/out? There is an SDK and server that help does that and file streaming is planned. But right now, you can execute commands in the VM and get the result back via the server
  - westurner 1y ago
    > I would consider Docker for VMs instead. Native Containers would probably solve here, too. From https://news.ycombinator.com/item?id=43553198 https://news.ycombinator.com/item?id=43553198 : >>> ostree native containers are bootable host images that can also be built and signed with a SLSA provenance attestation; https://coreos.github.io/rpm-ostree/container/ https://coreos.github.io/rpm-ostree/container/ And also from that thread: > How should a microkernel run (WASI) WASM runtimes? What is the most minimal microvm for WASM / WASI, and what are the advantages to running WASM workloads with firecracker or microsandbox?
    - appcypher 1y ago
      > What is the most minimal microvm for WASM / WASI, By setting up an image with wasmtime for example. > and what are the advantages to running WASM workloads with firecracker or microsandbox? I can think of stronger isolation or when you have legacy stuff you need to run alongside.
      - westurner 1y ago
        From https://e2b.dev/blog/firecracker-vs-qemu https://e2b.dev/blog/firecracker-vs-qemu > AWS built [Firecracker (which is built on KVM)] to power Lambda and Fargate [2], where they need to quickly spin up isolated environments for running customer code. Companies like E2B use Firecracker to run AI generated code securily in the cloud, while Fly.io uses it to run lightweight container-like VMs at the edge [4, 5]. "We replaced Firecracker with QEMU" (2023) https://news.ycombinator.com/item?id=36666782 https://news.ycombinator.com/item?id=36666782 "Firecracker's Kernel Support Policy" describes compatible kernel configurations; https://github.com/firecracker-microvm/firecracker/blob/main/docs/kernel-policy.md https://github.com/firecracker-microvm/firecracker/blob/main... /? wasi microvm kernel [github] https://www.google.com/search?q=wasi+microvm+kernel+GitHub https://www.google.com/search?q=wasi+microvm+kernel+GitHub : - "Mewz: Lightweight Execution Environment for WebAssembly with High Isolation and Portability using Unikernels" (2024) https://arxiv.org/abs/2411.01129 https://arxiv.org/abs/2411.01129 similar: https://scholar.google.com/scholar?q=related:b3657VNcyJ0J:scholar.google.com/&scioq=Mewz:+Lightweight+Execution+Environment+for+WebAssembly+with+High+Isolation+and+Portability+using+Unikernels&hl=en&as_sdt=0,43 https://scholar.google.com/scholar?q=related:b3657VNcyJ0J:sc...
hugs 1y ago
Looks great! This might be extremely useful for a distributed/decentralized software testing network I'm building (called Valet Network)... Question: How does networking work? Can I restrict/limit microvms so that they can only access public IP addresses? (or in other words... making sure the microvms can't access any local network IP addresses)
- appcypher 1y ago
  Yes! With the `scope` property. https://github.com/microsandbox/microsandbox/blob/0c13fc27ab687a28f56143bd3ce66769e840eb62/microsandbox-core/lib/config/microsandbox/config.rs#L222 https://github.com/microsandbox/microsandbox/blob/0c13fc27ab...
  - hugs 1y ago
    thanks! have an example on how to use that in a sandboxfile? (also, this project is really cool. great work!)
    - appcypher 1y ago
      Yeah. I need to fix that in the docs!
      - hugs 1y ago
        no prob!
simonw 1y ago
What's the story for macOS support?
- deleted 1y ago
  [deleted]
- appcypher 1y ago
  It uses libkrun which uses Hypervisor.framework on macOS.
wolfhumble 1y ago
Can you use Microsandbox for everything you can use Docker for, or are there cases where containers make more sense? Congratulations on the launch!
- appcypher 1y ago
  We want microsandbox to be usable for everything you can with Docker. That said, hosting microVMs require dedicated hardware or VMs with nested virt support. Containers don’t have that problem.
nqzero 1y ago
i'm on a mid-level laptop, at times with slow or expensive internet, running ubuntu. i want to be able to run nominally-isolated "copies" of my laptop at near-native speed 1. each one should have it's own network config, eg so i can use wireguard or a vpn 2. gui pass-through to the host, eg wayland, for trusted tools, eg firefox, zoom or citrix 3. needs to be lightweight. eg gnome-boxes is dead simple to setup and run and it works, but the resource usage was noticeably higher than native 4. optional - more security is better (ie, i might run semi-untrusted software in one of them, eg from a github repo or npm), but i'm not expecting miracles and accept that escape is possible 5. optional - sharing disk with the host via COW would be nice, so i'd only need to install the env-specific packages, not the full OS i'm currently working on a podman solution, and i believe that it will work (but rebuilding seems to hammer the network - i'm hoping i can tweak the layers to reduce this). does microsandbox offer any advantages for this use case ?
- appcypher 1y ago
  > 1. each one should have it's own network config, eg so i can use wireguard or a vpn This is possible right now but the networking is not where I want it to be yet. It uses libkrun's default TSI impl; performant and simplifies setup but can be inflexible. I plan to implement an alternative user-space networking stack soon. > 2. gui pass-through to the host, eg wayland, for trusted tools, eg firefox, zoom or citrix We don't have GUI passthrough. VNC? > 3. needs to be lightweight. eg gnome-boxes is dead simple to setup and run and it works, but the resource usage was noticeably higher than native It is lightweight in the sense that it is not a full vm > 4. optional - more security is better (ie, i might run semi-untrusted software in one of them, eg from a github repo or npm), but i'm not expecting miracles and accept that escape is possible The security guarantees are similar to what typical VMs support. It is hardware-virtualized so I would say you should be fine. > 5. optional - sharing disk with the host via COW would be nice, so i'd only need to install the env-specific packages, not the full OS Yeah. It uses virtio-fs and has overlayfs on top of that for COW.
simonw 1y ago
I'm trying this out now and it's very promising. One problem I'm running into with the Python library is that I'd like to keep that sandbox running for several minutes while I do things like set variables in one call and then use them for stuff several calls later. I keep seeing this error intermittently: Error: Sandbox is not started. Call start() first Is there a suggested way of keeping a sandbox around for longer? The documented code pattern is this: async def main(): async with PythonSandbox.create(name="my-sandbox") as sb: exec = await sb.run("print('Hello, World!')") print(await exec.output()) Due to the way my code works I want to instantiate the sandbox once for a specific class and then have multiple calls to it by class methods, which isn't a clean fit for that "async with" pattern. Any recommendations?
- appcypher 1y ago
  Right. You can skip the `with` context manager and call start and stop yourself. There is an example of that here: https://github.com/microsandbox/microsandbox/blob/0c13fc27ab687a28f56143bd3ce66769e840eb62/sdk/python/examples/command.py#L129-L153 https://github.com/microsandbox/microsandbox/blob/0c13fc27ab...
- gcharbonnier 1y ago
  async with is just syntactic sugar. You could very well call __aenter__ and __aexit__ manually. You could also use an AsyncExitStack, call __aenter__ manually, then enter_async_context, and call aclose when you’re done. Since aclose method exists I guess this is not an anti-pattern. https://docs.python.org/3/library/contextlib.html#contextlib.AsyncExitStack https://docs.python.org/3/library/contextlib.html#contextlib...
codethief 1y ago
Hi appcypher, very cool project! Does the underlying MicroVM feature provide an OCI runtime interface, so that it could be used as a replacement for runc/crun in Docker/Podman?
- Nypro 1y ago
  No. Not yet. Would be nice to have
  - codethief 1y ago
    Thanks for your response! One more question: What syscalls do I need to have access to in order to run a MicroVM? I'm asking because ideally I'd like to run container workloads inside existing containers (self-hosted GitLab CI runners) whose configuration (including AppArmor) I don't control.
Hilift 1y ago
Are you ready for the deluge of networking questions for all the buck wild configurations?
- Nypro 1y ago
  Lol. I should brace for impact. Networking continues to be a pain but I'm open to suggestions.
catlifeonmars 1y ago
How does the microvm architecture compare with firecracker?
- appcypher 1y ago
  They are similar. We use libkrun under the hood. Firecracker team seems not to be interested in a macOS implementation
  - catlifeonmars 1y ago
    Ah gotcha! I was unaware that firecracker used KVM under the hood.
nulld3v 1y ago
Cool project. Off topic question: Are the images in the "Use Cases" section in the README from a real app? I like the clean UI design.
- appcypher 1y ago
  No they are not.
spicybright 1y ago
I like the idea. But when you say "bullet proof" security, there are exploits to break out of VMs that exist. Have you looked into those?
- appcypher 1y ago
  Will fix the docs
meander_water 1y ago
Can you explain how this compares to Kata Containers? [0] That also supports OCI to run microVMs. You can also choose different hypervisors such as firecracker to run it on. [0] https://katacontainers.io/ https://katacontainers.io/
- appcypher 1y ago
  Katacontainers is an interesting project. Microsandbox is a more opinionated project with a UX that focuses on getting up and running with microVMs quickly. I want this experience for Linux, macOS and Windows users. More importantly is making sandboxing really accessible to AI devs with `msb server`.
nikolamus 1y ago
Think I can build a notebook on top of this ? Jupyter client has been a pain to manage
- appcypher 1y ago
  Not sure what that entails. You can try and I can help along the way
int_19h 1y ago
This is very neat tech, but I think you might want to wait until you actually have Windows covered before making claims like https://github.com/microsandbox/microsandbox/blob/main/MSB_V_DOCKER.md#2-true-cross-platform-consistency https://github.com/microsandbox/microsandbox/blob/main/MSB_V...
- appcypher 1y ago
  What do you mean?

Tsarp 1y ago

Wow. This looks awesome. Can we build our own python sandbox using the sandboxfile spec? This is if I want to add my own packages. Would this be just having my own requirements file here - https://github.com/microsandbox/microsandbox/blob/main/MSB_V_DOCKER.md#3-unified-configuration-model-the-sandboxfile https://github.com/microsandbox/microsandbox/blob/main/MSB_V...

appcypher 1y ago
Thank you! > Can we build our own python sandbox using the sandboxfile spec? Yes and I plan to make that work with the SDK. PS: Multi-stage build is WIP.
- Tsarp 1y ago
  Great will join the discord. Is this embeddable? Will it work with a cross platform desktop app(Tauri)?
  - deleted 1y ago
    [deleted]
  - apitman 1y ago
    An embeddable library that lets you launch Linux VMs that works across Windows, MacOS, and Linux hosts would be incredible.
  - appcypher 1y ago
    If by embeddable, you mean having the vm run in the same process, then no. The vm aborts its process when it's done so it has to run as separate process.

deleted 1y ago

[deleted]

jauntywundrkind 1y ago

Why not some of the existing microvm efforts? Cloud Hypervisor and Firecracker both have an excellent reputation for ultra lightweight VM's. Both are usable in the very popular Kata Containers project (as well as other upstart VM's Dragonball, & StratoVirt). In us by for example the CNCF Confidential Containers https://github.com/kata-containers/kata-containers/blob/main/docs/hypervisors.md#types https://github.com/kata-containers/kata-containers/blob/main... https://confidentialcontainers.org/ https://confidentialcontainers.org/ There's also smaller efforts such as firecracker-containerd or Virtink, both which bring OCI powered microvms into a Docker like position (easy to slot into Kubernetes), via Firecracker and Cloud Hypervisor respectively. https://github.com/smartxworks/virtink https://github.com/smartxworks/virtink https://github.com/firecracker-microvm/firecracker-containerd https://github.com/firecracker-microvm/firecracker-container... Poking around under the hood, microsandbox appears to use krun. There is krunvm for OCI support (includes MacOS/arm64 support!). https://github.com/containers/krunvm https://github.com/containers/krunvm https://github.com/slp/krun https://github.com/slp/krun The orientation as a safe sandbox for AI / MCP tools is a very nicely packaged looking experience, and very well marketred. Congratulations! I'm still not sure why this warrants being it's own project.

simonw 1y ago
If we get enough of these sandboxes, maybe we will finally get one that's easy for me to run on my own machines.
- mike_hearn 1y ago
  Which platforms do you use?
  - simonw 1y ago
    macOS on my laptop, anything that runs in a container for when I deploy things.
    - tough 1y ago

dataflow 1y ago

Tangential question: why does it normally take so long to start traditional VMs in the first place? At least on Windows, if you start a traditional VM, it takes several seconds for it to start running anything. Edit: when I say anything, I'm not talking user programs. I mean as in, before even the first instruction of the firmware -- before even the virtual disk file is zeroed out, in cases where it needs to be. You literally can't pause the VM during this interval because the window hasn't even popped up yet, and even when it has, you still can't for a while because it literally hasn't started running anything. So the kernel and even firmware initialization slowness are entirely irrelevant to my question. Why is that?

diggan 1y ago
I mean it is basically booting a computer from scratch, kind of makes sense. You have to allocate memory, start virtual CPUs, initialize devices, run BIOS/UEFI checks, perform hardware enumeration, all that jazz while emulating all of it, which tends to be slower than "real" implementations. I guess there is a bunch of processes for security as well, like wiping like zeroing pages and similar things that takes additional time. If I let a VM use most of my hardware, it takes a few seconds from start to login prompt, which is the same time it takes for my Arch desktop to boot from pressing the button to seeing the login prompt.
- dataflow 1y ago
  > You have to allocate memory, start virtual CPUs, initialize devices, run BIOS/UEFI checks, perform hardware enumeration, all that jazz while emulating all of it, which tends to be slower than "real" implementations. That's not what I'm asking. I'm saying it takes a long time for it to even execute a single instruction, in the BIOS itself. Even for the window to pop up, before you can even pause the VM (because it hasn't even started yet). What you're describing comes after all that, which I already understand and am not asking about.
  - drewg123 1y ago
    Without any context in terms of what the VM is doing or what VMM software you use, my best guess is that the OS/VMM are pre-allocating memory for the VM. This might involve paging out other processes' memory, which could take some time. I think task manager would tell you if there is a blip of memory usage and paging activity at the time. And I'm sure windows itself has profilers that can tell you what is happening when the VM is started..

Jayakumark 1y ago

Windows support ? and can we VNC in to the sandbox and stream it ?

appcypher 1y ago
Windows support is a work in progress. I haven't tested using VNC yet but it should be possible.
spixy 1y ago
Windows already has built-in sandbox and its good. https://learn.microsoft.com/en-us/windows/security/application-security/application-isolation/windows-sandbox/ https://learn.microsoft.com/en-us/windows/security/applicati...

h1fra 1y ago

Can't wait to test, if it's really what's advertised it would be much easier to use than workerd or firecracker

McAlpine5892 1y ago

This looks awesome. The amount of super lightweight and almost-disposable VM options in recent years is crazy. I remember when VMs were slow, clunky, and generally painful. I wonder how this compares to Orbstack's [0] tech stack on macOS, specifically the "Linux machines" [1] feature. Seems like Orb might reuse a single VM? --- [0] https://orbstack.dev https://orbstack.dev [1] https://docs.orbstack.dev/machines/ https://docs.orbstack.dev/machines/

jbverschoor 1y ago

Related, https://github.com/jrz/container-shell https://github.com/jrz/container-shell which uses docker to create adhoc shells / chroots in the current directory.

manveru 1y ago

Are the SDKs AI generated? I looked at the Crystal, Ruby, and Zig ones and all they contain is a hello world example with some docs that have little to do with the code. Sorry if this comment seems rude, just curious.

appcypher 1y ago
The other SDKs are generated hello-worlds at the moment. I will get to them one by one, but I welcome and appreciate any contributions to them.

jmehman 1y ago

I've been looking for something I could host for this kind of thing - for LLM agents. Ended up on https://www.daytona.io/ https://www.daytona.io/ as I couldn't find anything suitable to self host and realised it was a complex thing to manage. It seems Daytona is open source, including the server platform, but there is no documentation for the server element. Azure also seem to offer a service for this, it's a space that is growing rapidly.

appcypher 1y ago
Microsandbox is for people that would like to maintain their own infra. I'm not going to stop trying to make it better to self-host.
- jmehman 1y ago
  Yeah, it looks great, makes me reconsider the self hosted route

patrick4urcloud 1y ago

very nice ! i will definetly try

ATechGuy 1y ago

Congrats on launching! Booting VMs in milliseconds is certainly important, but it can also be achieved with CloudHypervisor/Firecracker. Where Containers beat VMs is runtime perf. The overhead in case of VMs stems from emulation of IO devices. I believe the overhead will become noticeable for AI agentic use cases. Any plans to address perf issues?

appcypher 1y ago
You are right. We leverage libkrun. Libkrun uses virtio-mmio transport for block, vsock and virtio-fs to keep overhead minimal so we basically depend on any perf improvement made upstream. Firecracker is no different btw and E2B uses that for agentic AI workloads. Anyway, I don't have any major plan except fix some issues with the filesystem rn.

SwiftyBug 1y ago

Kind of almost off-topic: I'm working on a project where I must run possibly untrusted JavaScript code. I want to run it in an isolated environment. This looks like a very nice solution as I could spin up a microsandbox and securely run the code. I could even have a pool os live sandboxes so I wouldn't even experience the 200ms starts. Because this is OCI-compatible, I could even provide a whole sandboxed environment on which to run that code. Would that be a good use case for this? Are there better alternatives?

appcypher 1y ago
> Would that be a good use case for this? That is an ideal use case > Are there better alternatives? Created microsandbox because I didn't find any
- SwiftyBug 1y ago
  Awesome. This is really good timing. I'm going to give it a try.
- spixy 1y ago
  Windows has built-in sandbox and its good. https://learn.microsoft.com/en-us/windows/security/application-security/application-isolation/windows-sandbox/ https://learn.microsoft.com/en-us/windows/security/applicati...
ericb 1y ago
runsc / gVisor is interesting also as the runsc engine can be run from within Docker/Docker Desktop. gVisor has performance problems, though. Their data shows 1/3rd the throughput vs. docker runtime for concurrent network calls--if that's an issue for your use-case.
apitman 1y ago
You might be able to get away with running QuickJS compiled to WebAssembly: https://til.simonwillison.net/npm/self-hosted-quickjs https://til.simonwillison.net/npm/self-hosted-quickjs

deleted 1y ago

[deleted]

hinkley 1y ago

How’s performance? What’s the overhead versus docker? Terraform or Pulumi integration on the horizon?

appcypher 1y ago
Wow. Just seeing this. I've not done proper benchmarking yet but rn we are lagging behind in file I/O for the OverlayFS impl
- hinkley 1y ago
  There was a period where NFS was faster, particularly on windows and OSX where you were paying a double indirection. Overlays are always tough because docker doesn’t like you writing to the filesystem in the first place. The weapon if first result is deflection; tell them not to do it. I had to put up with an old docker version that leaked overlay data for quite a while before we moved off prem.

elwebmaster 1y ago

One topic I am not finding anything about is networking. Can these microsandbox instances listen on ports? How is the port forwarding configured? Can they access the internet or any resources on the host?

appcypher 1y ago
They can. I need to improve the doc. Working on that right now

zackmorris 1y ago

This is great! I'd like to see a formal container security grade that works like: 1) Curate a list of all known (container) exploits 2) Run each exploit in environments of increasing security like permissions-based, jail, Docker and emulator 3) The percentage of prevented exploits would be the score from 0-100% Under this scheme, I'd expect naive attempts at containerization with permissions and jails to score around 0%, while Docker might be above 50% and Microsandbox could potentially reach 100%. This might satisfy some of our intuition around questions like "why not just use a jail?". Also the containers could run on a site on the open web as honeypots with cash or crypto prizes for pwning them to "prove" which containers achieve 100%. We might also need to redefine what "secure" means, since exploits like Rowhammer and Spectre may make nearly all conventional and cloud computing insecure. Or maybe it's a moving target, like how 64 bit encryption might have once been considered secure but now we need 128 bit or higher. Edit: the motivation behind this would be to find a container that's 100% secure without emulation, for performance and cost-savings benefits, as well as gaining insights into how to secure operating systems by containerizing their various services.

bjackman 1y ago
You cannot build a secure container runtime (against malicious containers) because underlying it is the Linux kernel. The only way to make Linux containers a meaningful sandbox is to drastically restrict the syscall API surface available to the sandboxee, which quickly reduces its value. It's no longer a "generic platform that you can throw any workload onto" but instead a bespoke thing that needs to be tuned and reconfigured for every usecase. This is why you need virtualization. Until we have a properly hardened and memory safe OS, it's the only way. And if we do build such an OS it's unclear to me whether it will be faster than running MicroVMs on a Linux host.
- Veserv 1y ago
  You cannot build a secure virtualization runtime because underlying it is the VMM. Until you have a secure VMM you are subject to precisely the same class of problems plaguing container runtimes. The only meaningful difference is that Linux containers target partitioning Linux kernel services which is a shared-by-default/default-allow environment that was never designed for and has never achieved meaningful security. The number of vulnerabilities resulting from, "whoopsie, we forgot to partition shared service 123" would be hilarious if it were not a complete lapse of security engineering in a product people are convinced is adequate for security-critical applications. Present a vulnerability assessment demonstrating a team of 10 with 3 years time (~10-30 M$, comparable to many commercially-motivated single-victim attacks these days) can find no vulnerabilities in your deployment or a formal proof of security and correctness otherwise we should stick with the default assumption that software if easily hacked instead of the extraordinary claim that demands extraordinary evidence.

rbitar 1y ago

Looks great and excited to try this out. We’ve also had success using CodeSandbox SDK and E2B, can you share some thoughts on how you compare or future direction? Do you also use Firecracker under the hood?

pkkkzip 1y ago
I can't tell if it uses firecracker but thats my main question too. I'm curious as to whether microsandbox will be maintained and proper auditing will be done. I welcome alternatives. It's been tough wrestling with Firecracker and OCI images. Kata container is also tough.
- appcypher 1y ago
  It will be maintained as I will be using it for some other product. And it will be audited in the future but it still early days.
- pdimitar 1y ago
  I wanted to try Kata containers soon. What difficulties do you have with them?
appcypher 1y ago
> can you share some thoughts on how you compare or future direction? Microsandbox does not offer a cloud solution. It is self-hosted, designed to do what E2B does, to make it easier working with microVM-based sandboxes on your local machine whether that is Linux, macOS or Windows (planned) and to seamlessly transition to prod. > Do you also use Firecracker under the hood? It uses libkrun.
- rbitar 1y ago
  Self-hosting is definitely something we are keen to explore as most of the cloud solutions have resource constrains (ie, total active MicroVMs and/or specs per VM) and managing billing gets complicated even with hibernation features. Great project and we'll definitely take it for a spin

sureglymop 1y ago

Always interested when things like this come up. What like about containers is how quickly I can run something, e.g. `docker run --rm ...` without having to specify disk size, amount of cpu cores, etc. I can then diff the state of the container with the image (and other things) to see what some program did while it ran. So I basically want the same but instead with small vms to have better sandboxing. Sometimes I also use bwrap but it's not really intended to be used on the command line like that.

srmatto 1y ago
It has a YAML config format to declare all of that so you could just do that once, or template it, generate it on the fly, fetch it from remote, or many other methods.

eamann 1y ago

> Ever needed to run code you don't fully trust? Then the installation instructions include piping a remote script directly to Bash ... Oh irony ... That said, the concept itself is intriguing.

deleted 1y ago
[deleted]
appcypher 1y ago
Your statement initially went over my head. Sorry lol. You can always download the installer script and audit yourself. I will set up proper distribution later.
- hakcermani 1y ago
  .. did exactly that and also changed the BINDIR and LIBDIR to another location. BTW, amazing project from initial glance. Will give it a detailed look this weekend!
- raphinou 1y ago
  In case you're interested when you set up proper distribution, I'm working on an open source solution aiming to improve security of downloads from the internet. Our first step is maintaining a mirror of checksums published in GitHub releases at https://github.com/asfaload/checksums/ https://github.com/asfaload/checksums/. If you publish a checksums file in your releases it can automatically be mirrored. The checksums mirror is not our end game, but it already protects against changes of released files from the time the mirror was taken. For anyone interested: https://asfaload.com/asfald/ https://asfaload.com/asfald/

amelius 1y ago

For my taste, container technology is pushing the OS too far. By typing: mount you immediately see what I mean. Stuff that should be hidden is now in plain sight, and destroys the usefulness of simple system commands. And worse, the user can fiddle with the data structures. It's like giving the user peek and poke commands. The idea of containers is nice, but they are a hack until kernels are re-architected.

throwaway314155 1y ago
Sorry I am lacking the context to understand this post. What does running mount inside a container do that's so egregious? Are host mounts exposed to the container somehow? I thought everything needed to be explicitly passed through to the container (e.g. using a volume)?
- remram 1y ago
  I think they mean that running `mount` on the host now lists hundreds of mountpoints from containers, snaps, packagekit etc.
topspin 1y ago
On recent Linux, try: findmnt --real It's part of linux-utils, so it is generally available wherever have a shell. The legacy tools you have in mind aren't ever going to be changed as you would wish, for reasons.

sbassi 1y ago

There are python and node environment for this, so they are not VMs in the sense that I can host a OS and arbitrary executables?

appcypher 1y ago
They are Linux VMs and you can host any executable that can work on that. The python/node environment you see is part of what makes the SDK work. Really, it's very similar to Docker in use.
- sbassi 1y ago
  thank you. Is there any "docker host" or centralized repo where I can pull VMs from?
  - appcypher 1y ago
    We support just Docker hub for now. Let me know if you want any other OCI-compatible registry. PS: microsandbox will likely have its own OCI registry in the future

airocker 1y ago

Would love to hear nix people take on this?

mjrusso 1y ago
As a Nix user, I'm actually really excited to try this out. I want to run sandboxes based on Docker images that have Nix pre-installed. (Once the VM boots, apply the project-specific Flake, and then run Docker Compose for databases and other supporting services.) In theory, an easy-to-use, fully isolated dev environment that matches how I normally develop, except inside of a VM.
- airocker 1y ago
  but dont they have overlapping requirements of solving "not works on my machine"
  - mjrusso 1y ago
    Microsandbox's primary goal is to make it easy to build environments for running untrusted code. Nix, on the other hand, solves the problem of building reproducible environments... but making said environments safe for running untrusted code is left as an exercise for the reader.