6 ms·

Hello, I'm building a replacement for docker containers with a virtual machine with the ergonomics of containers + subsecond start times. I worked in AWS previ

Hello, I'm building a replacement for docker containers with a virtual machine with the ergonomics of containers + subsecond start times. I worked in AWS previously in the container space + with firecracker. I realized the container is an unnecessary layer that slowed things down + firecracker was a technology designed for AWS org structure + usecase. So I ended up building a hybrid taking the best of containers with the best of firecracker. Let me know your thoughts, thanks!

harshdoesdev 6mo ago
+1. i built something similar called shuru.run because i wanted an easy way to set up microVM sandboxes to run some of my AI apps, and firecracker wasn't available for macOS (and, as you said, it is just too heavy for normal user-level workloads).
- fqiao 6mo ago
  Yes, having a light-weight solution for local devices as well is one primary goal of the design. Another one is to make it easy for hosting, self or managed
- sahil-shubham 6mo ago
  Nice work on Shuru — I remember looking at it when I was researching this space. You went with a Rust wrapper on Apple’s Virtualization framework right? I have been working on something similar but on top of firecracker, called it bhatti (https://github.com/sahil-shubham/bhatti https://github.com/sahil-shubham/bhatti). I believe anyone with a spare linux box should be able to carve it into isolated programmable machines, without having to worry about provisioning them or their lifecycle. The documentation’s still early but I have been using it for orchestrating parallel work (with deploy previews), offloading browser automation for my agents etc. An auction bought heztner server is serving me quite well :)
  - harshdoesdev 6mo ago
    bhatti's cli looks very ergonomic! great job! also, yes, shuru was (still) a wrapper over the Virtualization.framework, but it now supports Linux too (wrapper over KVM lol)
  - davidcollantes 6mo ago
    Is there a way to store configuration/data of applications running on a Bhatti VM on the host, ala Docker volumes?
    - sahil-shubham 6mo ago
      Yes! Checkout the bhatti volume They are ext4 blocks which exist independent of sandboxes.
      - davidcollantes 6mo ago
        Wow, that’s quite nice! I see that’s at VM time, any plans to add the ability to add them after the fact? I am really enjoying bhatti!
thm 6mo ago
You could add OrbStack to the comp. table
- fqiao 6mo ago
  Will do. Thanks for the suggestion!
sdrinf 6mo ago
hi, great project! Windows support is sorely lacking, though. As someone working a lot with sandboxed LLMs right now, the options-space on windows for sandboxing is _extremely lacking_. Any plans to support it?
- binsquare 6mo ago
  Yeah, it's in my mind. WSL2 runs a linux virtual machine. Need to take some time and care to wire that up, but definitely feasible.
- fqiao 6mo ago
  Hey, thanks so much! yah we will definitely add windows support later. We are exploring how to get this work with WSL and will release it asap. Stay tuned and thanks!
- xnx 6mo ago
  Not sure what level of sandboxing you need. Is Sandboxie not enough?
PufPufPuf 6mo ago
Hey this is super cool. I've been researching tech like this for my AI sandboxing solution, ended up with Lima+Incus: https://github.com/JanPokorny/locki https://github.com/JanPokorny/locki My problem with microVMs was that they usually won't run docker / kubernetes, I work on apps that consist of whole kubernetes clusters and want the sandbox to contain all that. Does your solution support running k3s for example?
- fqiao 6mo ago
  we will evaluate. I created this issue to track this: https://github.com/smol-machines/smolvm/issues/150 https://github.com/smol-machines/smolvm/issues/150 Really appreciate the feedback!
- deleted 6mo ago
  [deleted]
- mkagenius 6mo ago
  With instavm (https://instavm.io https://instavm.io), you can provide an OCI image built from a dockerfile.
  - PufPufPuf 6mo ago
    That's not the same as running the docker daemon in the sandbox.
- jaytheseveloper 6mo ago
  Curious what prevented you from running docker in a microvm? I've successfully run docker in qemu and firecracker microvms.
  - fqiao 6mo ago
    So we originally just wanted the vm to be light-weight and can start fast, so we removed docker. @binsquare has a change later to add docker back.
  - PufPufPuf 6mo ago
topspin 6mo ago
What is the status of supporting live migration? That's the one feature of similar systems that always gets left out. I understand why: it's not a priority for "cloud native" workloads. The world, however, has work loads that are not cloud native, because that comes at a high cost, and it always will. So if you'd like a real value-add differentiator for your micro-VM platform (beyond what I believe you already have,) there you go. Otherwise this looks pretty compelling.
- fqiao 6mo ago
  Really appreciate the suggestion! By "live migration", do you mean keeping the existing files and migrate them elsewhere with the vm? Thanks
  - topspin 6mo ago
    I mean making any given VM stop on host A and appear on host B; e.g. standard Qemu/KVM: virsh migrate --live GuestName DestinationURL This is feasible when network storage is available and useful when a host needs to be drained for maintenance.
    - fqiao 6mo ago
      I see. so right now smolvm can be stopped, and then "packed" (think of it as compressed), and restart on a different host. files in the disks are preserved, but memory snapshotting is still hard tbh
      - fragmede 6mo ago
        Ultimately the original does get stopped, but with additional techniques, we're talking milliseconds of downtime between when the old one stops and the new one resumes. (For live migration technology in general, no clue about smol machines.)
    - sureglymop 6mo ago
      It's also feasible without network storage, --copy-storage-all will migrate all disks too.
lacoolj 6mo ago
What percentage of this code was written by LLM/AI?
- binsquare 6mo ago
  For myself, I'd estimate ~50% Not useful for things it hadn't been trained on before. But now I have the core functionality in place - it's been of great help.
- RALaBarge 6mo ago
  [flagged]
  - anthk 6mo ago
    Hey 'software engineer', how much of the output of an LLM it's actually reproducible vs the one from a calculator or any programming language with the same input in different sessions?
    - weird-eye-issue 6mo ago
      Why are you so concerned about the LLM producing the exact same code across different sessions? Seems like a really weird thing to focus on. Why aren't you focused on things like security, maintainability, UI/UX, performance?
      - linsomniac 6mo ago
        Agreed. It's not like humans can produce the same output given the same input for anything more than trivial inputs. I'd argue that it's actually a benefit; I like that I can do several generations and compare them and pick the best result. HP, for example, used to do this with software teams, and that's how we got Rocky Mountain BASIC (AIUI the competing team was East Coast BASIC).
      - naikrovek 6mo ago
        Comp-sci people like repeatability when they want that and true randomness when that is desired. Things in between are rarely desired. In computing, things are much more useful when they behave in predictable ways. Even AI, many (most?) would argue.
JuniperMesos 6mo ago
What were the biggest challenges in terms of designing the VM to have subsecond start times? And what are the current bottlenecks for deceasing the start time even further?
- binsquare 6mo ago
  No special programming tricks were used. Linux was built in the 90s. Hardware improved more than a 1000x. Linux virtual machine startup times stayed relatively the same. Turns out we kept adding junk to the linux kernel + bootup operations. So all I did was cut and remove unnecessary parts until it still worked. This ended up also getting boot up times to under 1s. The kernel changes are the 10 commits I made, you can verify here: https://github.com/smol-machines/libkrunfw https://github.com/smol-machines/libkrunfw There's probably more fat to cut to be honest.
thepoet 6mo ago
The images or rather portable artifacts rehydration on any platform plus the packaging is neat. I have been working on https://instavm.io https://instavm.io for some time around VM based sandboxes and related infra for agents and this is refreshing to see.
BobbyTables2 6mo ago
How is this different than Kara Containers?
- binsquare 6mo ago
  kata containers is a container runtime that focuses on running containers inside a vm. smolvm is a vm with some of the properties & ergonomics of containers - it's meant as a replacement for containers.
todotask2 6mo ago
Just curious if this allow to sync file changes from host to vm which only Docker has that and no other VM alternative has that ability.
- binsquare 6mo ago
  it has directory mounts - so access the same files: https://github.com/smol-machines/smolvm/blob/main/AGENTS.md?plain=1#L113 https://github.com/smol-machines/smolvm/blob/main/AGENTS.md?... It also has file copy from host to inside the guest: https://github.com/smol-machines/smolvm/blob/70c97930dda5a9288fbe4bdd0ac3109db9c31450/AGENTS.md?plain=1#L225 https://github.com/smol-machines/smolvm/blob/70c97930dda5a92...
rickydroll 6mo ago
How is this different from lxd/lxc? How tied is this to kvm or could it work with other hypervisors like xcp-ng, VMware, or virtual box?
- binsquare 6mo ago
  lxc/lxd are ~container managers, the kernel is still shared like docker containers. It runs on linux & needs to run in a linux virtual machine on macOS. smolvm is a virtual machine with the ergonomics of containers like packaging and distributing, kernel is not shared. You'd run containers inside of smolvm. Also, smolvm runs ontop of both kvm for linux and apple's hypervisor for macOS. So it's cross platform.
linsomniac 6mo ago
Any Windows support coming? My dev team is on Windows, this could be a game-changer.
- binsquare 6mo ago
  Wsl2 runs a Linux vm so it definitely feasible and has the api's necessary. It is on the roadmap, but frankly I haven't used Windows in a decade. I would love for a contributor to take that on as part of the free and open source spirit. Sounds like it could be you? :)
  - linsomniac 6mo ago
    >Sounds like it could be you? :) I wish it could, but, like your housekeeper, I don't do Windows. ;-)
gysakai 6mo ago
My impression from the `.smolmachine` single-binary packaging is that it could have a massive impact on next-gen edge computing and distributed serverless infrastructure. To me, it feels like it could fulfill WASM's promise of a "portable execution unit that runs instantly anywhere," but with the added benefit of keeping the existing Linux binary ecosystem intact via microVMs. I'm curious—is this kind of use case aligned with the direction you're aiming for?
- binsquare 6mo ago
  Yes, the exact use case I'm targeting.

harshdoesdev 6mo ago

+1. i built something similar called shuru.run because i wanted an easy way to set up microVM sandboxes to run some of my AI apps, and firecracker wasn't available for macOS (and, as you said, it is just too heavy for normal user-level workloads).

fqiao 6mo ago
Yes, having a light-weight solution for local devices as well is one primary goal of the design. Another one is to make it easy for hosting, self or managed
sahil-shubham 6mo ago
Nice work on Shuru — I remember looking at it when I was researching this space. You went with a Rust wrapper on Apple’s Virtualization framework right? I have been working on something similar but on top of firecracker, called it bhatti (https://github.com/sahil-shubham/bhatti https://github.com/sahil-shubham/bhatti). I believe anyone with a spare linux box should be able to carve it into isolated programmable machines, without having to worry about provisioning them or their lifecycle. The documentation’s still early but I have been using it for orchestrating parallel work (with deploy previews), offloading browser automation for my agents etc. An auction bought heztner server is serving me quite well :)
- harshdoesdev 6mo ago
  bhatti's cli looks very ergonomic! great job! also, yes, shuru was (still) a wrapper over the Virtualization.framework, but it now supports Linux too (wrapper over KVM lol)
- davidcollantes 6mo ago
  Is there a way to store configuration/data of applications running on a Bhatti VM on the host, ala Docker volumes?
  - sahil-shubham 6mo ago
    Yes! Checkout the bhatti volume They are ext4 blocks which exist independent of sandboxes.
    - davidcollantes 6mo ago
      Wow, that’s quite nice! I see that’s at VM time, any plans to add the ability to add them after the fact? I am really enjoying bhatti!

thm 6mo ago

You could add OrbStack to the comp. table

fqiao 6mo ago
Will do. Thanks for the suggestion!

sdrinf 6mo ago

hi, great project! Windows support is sorely lacking, though. As someone working a lot with sandboxed LLMs right now, the options-space on windows for sandboxing is _extremely lacking_. Any plans to support it?

binsquare 6mo ago
Yeah, it's in my mind. WSL2 runs a linux virtual machine. Need to take some time and care to wire that up, but definitely feasible.
fqiao 6mo ago
Hey, thanks so much! yah we will definitely add windows support later. We are exploring how to get this work with WSL and will release it asap. Stay tuned and thanks!
xnx 6mo ago
Not sure what level of sandboxing you need. Is Sandboxie not enough?

PufPufPuf 6mo ago

Hey this is super cool. I've been researching tech like this for my AI sandboxing solution, ended up with Lima+Incus: https://github.com/JanPokorny/locki https://github.com/JanPokorny/locki My problem with microVMs was that they usually won't run docker / kubernetes, I work on apps that consist of whole kubernetes clusters and want the sandbox to contain all that. Does your solution support running k3s for example?

fqiao 6mo ago
we will evaluate. I created this issue to track this: https://github.com/smol-machines/smolvm/issues/150 https://github.com/smol-machines/smolvm/issues/150 Really appreciate the feedback!
deleted 6mo ago
[deleted]
mkagenius 6mo ago
With instavm (https://instavm.io https://instavm.io), you can provide an OCI image built from a dockerfile.
- PufPufPuf 6mo ago
  That's not the same as running the docker daemon in the sandbox.
jaytheseveloper 6mo ago
Curious what prevented you from running docker in a microvm? I've successfully run docker in qemu and firecracker microvms.
- fqiao 6mo ago
  So we originally just wanted the vm to be light-weight and can start fast, so we removed docker. @binsquare has a change later to add docker back.
- PufPufPuf 6mo ago

topspin 6mo ago

What is the status of supporting live migration? That's the one feature of similar systems that always gets left out. I understand why: it's not a priority for "cloud native" workloads. The world, however, has work loads that are not cloud native, because that comes at a high cost, and it always will. So if you'd like a real value-add differentiator for your micro-VM platform (beyond what I believe you already have,) there you go. Otherwise this looks pretty compelling.

fqiao 6mo ago
Really appreciate the suggestion! By "live migration", do you mean keeping the existing files and migrate them elsewhere with the vm? Thanks
- topspin 6mo ago
  I mean making any given VM stop on host A and appear on host B; e.g. standard Qemu/KVM: virsh migrate --live GuestName DestinationURL This is feasible when network storage is available and useful when a host needs to be drained for maintenance.
  - fqiao 6mo ago
    I see. so right now smolvm can be stopped, and then "packed" (think of it as compressed), and restart on a different host. files in the disks are preserved, but memory snapshotting is still hard tbh
    - fragmede 6mo ago
      Ultimately the original does get stopped, but with additional techniques, we're talking milliseconds of downtime between when the old one stops and the new one resumes. (For live migration technology in general, no clue about smol machines.)
  - sureglymop 6mo ago
    It's also feasible without network storage, --copy-storage-all will migrate all disks too.

lacoolj 6mo ago

What percentage of this code was written by LLM/AI?

binsquare 6mo ago
For myself, I'd estimate ~50% Not useful for things it hadn't been trained on before. But now I have the core functionality in place - it's been of great help.
RALaBarge 6mo ago
[flagged]
- anthk 6mo ago
  Hey 'software engineer', how much of the output of an LLM it's actually reproducible vs the one from a calculator or any programming language with the same input in different sessions?
  - weird-eye-issue 6mo ago
    Why are you so concerned about the LLM producing the exact same code across different sessions? Seems like a really weird thing to focus on. Why aren't you focused on things like security, maintainability, UI/UX, performance?
    - linsomniac 6mo ago
      Agreed. It's not like humans can produce the same output given the same input for anything more than trivial inputs. I'd argue that it's actually a benefit; I like that I can do several generations and compare them and pick the best result. HP, for example, used to do this with software teams, and that's how we got Rocky Mountain BASIC (AIUI the competing team was East Coast BASIC).
    - naikrovek 6mo ago
      Comp-sci people like repeatability when they want that and true randomness when that is desired. Things in between are rarely desired. In computing, things are much more useful when they behave in predictable ways. Even AI, many (most?) would argue.

JuniperMesos 6mo ago

What were the biggest challenges in terms of designing the VM to have subsecond start times? And what are the current bottlenecks for deceasing the start time even further?

binsquare 6mo ago
No special programming tricks were used. Linux was built in the 90s. Hardware improved more than a 1000x. Linux virtual machine startup times stayed relatively the same. Turns out we kept adding junk to the linux kernel + bootup operations. So all I did was cut and remove unnecessary parts until it still worked. This ended up also getting boot up times to under 1s. The kernel changes are the 10 commits I made, you can verify here: https://github.com/smol-machines/libkrunfw https://github.com/smol-machines/libkrunfw There's probably more fat to cut to be honest.

thepoet 6mo ago

The images or rather portable artifacts rehydration on any platform plus the packaging is neat. I have been working on https://instavm.io https://instavm.io for some time around VM based sandboxes and related infra for agents and this is refreshing to see.

BobbyTables2 6mo ago

How is this different than Kara Containers?

binsquare 6mo ago
kata containers is a container runtime that focuses on running containers inside a vm. smolvm is a vm with some of the properties & ergonomics of containers - it's meant as a replacement for containers.

todotask2 6mo ago

Just curious if this allow to sync file changes from host to vm which only Docker has that and no other VM alternative has that ability.

binsquare 6mo ago
it has directory mounts - so access the same files: https://github.com/smol-machines/smolvm/blob/main/AGENTS.md?plain=1#L113 https://github.com/smol-machines/smolvm/blob/main/AGENTS.md?... It also has file copy from host to inside the guest: https://github.com/smol-machines/smolvm/blob/70c97930dda5a9288fbe4bdd0ac3109db9c31450/AGENTS.md?plain=1#L225 https://github.com/smol-machines/smolvm/blob/70c97930dda5a92...

rickydroll 6mo ago

How is this different from lxd/lxc? How tied is this to kvm or could it work with other hypervisors like xcp-ng, VMware, or virtual box?

binsquare 6mo ago
lxc/lxd are ~container managers, the kernel is still shared like docker containers. It runs on linux & needs to run in a linux virtual machine on macOS. smolvm is a virtual machine with the ergonomics of containers like packaging and distributing, kernel is not shared. You'd run containers inside of smolvm. Also, smolvm runs ontop of both kvm for linux and apple's hypervisor for macOS. So it's cross platform.

linsomniac 6mo ago

Any Windows support coming? My dev team is on Windows, this could be a game-changer.

binsquare 6mo ago
Wsl2 runs a Linux vm so it definitely feasible and has the api's necessary. It is on the roadmap, but frankly I haven't used Windows in a decade. I would love for a contributor to take that on as part of the free and open source spirit. Sounds like it could be you? :)
- linsomniac 6mo ago
  >Sounds like it could be you? :) I wish it could, but, like your housekeeper, I don't do Windows. ;-)

gysakai 6mo ago

My impression from the `.smolmachine` single-binary packaging is that it could have a massive impact on next-gen edge computing and distributed serverless infrastructure. To me, it feels like it could fulfill WASM's promise of a "portable execution unit that runs instantly anywhere," but with the added benefit of keeping the existing Linux binary ecosystem intact via microVMs. I'm curious—is this kind of use case aligned with the direction you're aiming for?

binsquare 6mo ago
Yes, the exact use case I'm targeting.