12 ms·

Show HN: Ephemeral VMs in 1 Microsecond

84 points by fwsgonzo 2y ago

kjok 2y ago
What are the use cases?
- selcuka 2y ago
  Explained in the first and second paragraphs of the README: > Multi-tenancy allows one server to be safely shared among many users, each of which cannot access each others or negatively affect the HTTP service. > Specialized sandboxes are instantiated for each request and immediately destroyed after the request, all within a single microsecond.
  - goodpoint 2y ago
    Some more practical examples would be nice, including security impact
pyaamb 2y ago
Whatever happened to Unikernels?
- lgas 2y ago
  Nothing.
- yla92 2y ago
  Still around. Recently, Prisma launched their managed Postgres offering built on top of Unikernels https://www.prisma.io/blog/announcing-prisma-postgres-early-access https://www.prisma.io/blog/announcing-prisma-postgres-early-...
- goodpoint 2y ago
  They are still in the same tiny niche
mdaniel 2y ago
This GitHub org was cited several times in the recent "source available" annoucement for MoonBit, if one wishes to see more "in the wild" usage https://news.ycombinator.com/item?id=42450274 https://news.ycombinator.com/item?id=42450274
thornewolf 2y ago
to check my understanding on what this is offering, I could build something on top of this that offers remote code execution for people without needing to worry about my system being compromised? or other people's processes interacting with one another, but the VM will still be able to make web requests itself?
- fwsgonzo 2y ago
  The VM could only make requests if you add a system call specifically for that purpose. Eg. if you add something like sys_request that takes a URL as argument and some input and output arguments. It's not that the VM couldn't open a socket and handle connections by itself, but it's all closed down. I think when it comes to being integrated into a web server, you ideally want to use the web servers pipeline to make requests, so that you can benefit from the same logging and observability that Drogon (or other solutions) already have. If you allow the VM to make requests on its own, you just have the IP and an open socket. Not much to debug with if (let's say) you have a customer asking about something.
jumploops 2y ago
What do VMs mean in this context? I did a pass of the codebase and it seems they’re just forking processes? It’s unclear to me where the safety guarantees come from (compared to using e.g. KVM). Edit: it appears the safety guarantees come from libriscv[0]. As far as I can tell, these sandboxes are essentially RISC-V programs running in an isolated context (“machine”) where all the Linux syscalls are emulated and thus “safe.” Still curious what potential attack vectors may exist? [0] https://github.com/libriscv/libriscv/tree/dfb7c85d01f01cb38fb34b529894a2baaa902d04 https://github.com/libriscv/libriscv/tree/dfb7c85d01f01cb38f...
- RainyDayTmrw 2y ago
  See also: Google's GVisor [1], which was used in Google App Engine. [1]: https://en.wikipedia.org/wiki/GVisor https://en.wikipedia.org/wiki/GVisor
- fwsgonzo 2y ago
  Well, the boundary between the host and the guest, the system call API, is always going to be the biggest vector of attacks no matter what the solution used is. But, if you find a problem and fix it, you're back to being safe again, unlike if you don't have any sandboxing at all. You can also put the whole solution in a jail, which is very common nowadays.
  - jumploops 2y ago
    Can you expand on “put the whole solution in a jail”?
    - fwsgonzo 2y ago
      FireCracker has a jailer: https://github.com/firecracker-microvm/firecracker/blob/main/docs/jailer.md https://github.com/firecracker-microvm/firecracker/blob/main...
      - jumploops 2y ago
        Ah, this is helpful, thanks!
mattclarkdotnet 2y ago
Having scanned the codebase, I think this is about quickly and safely launching and managing risc-v binaries as sandboxed processes? Which is useful, but has nothing to do with virtual machines in the usual sense of there being a hypervisor with hardware support for isolation.
EgoIncarnate 2y ago
The use of the term VM without further qualification in the title is unfortunate. Emulated VM would have been nicer to avoid confusion with hypervisor style virtual machines. Staring emphermial hypervisor VMs quickly is more noteworthy (since they are often slow to start) than an emulator VM where it's expected to be fast since it's usually not much more than setting up a datastructure and executing a call to an interpreter. I clicked hoping for the former, only to find out the project is the latter.
- binary132 2y ago
  What exactly is the difference? Are you talking about hardware virtualization via e.g. intel vt-x? Do you mean virtualizing hardware subsystems with drivers instead of forwarding syscalls directly? Running a kernel? Maybe I’m not seeing how those things are fundamentally different than “setting up a datastructure and calling an interpreter”.
  - EgoIncarnate 2y ago
    A hypervisor VM is running native code via hardware virtualization. An emulator VM is running an interpreter and/or JIT generally using a different instruction set. A hypervisor VM typically requires more extensive setup involving hardware configuration and is usually used for running existing native code, so it often means emulating a real machine, including OS and sometimes even firmware/BIOS. There are "lightweight" environments like Firecracker, but the overhead of creating an instance is still heaver than the overhead of a function call. The instance creation overhead is high, but the instruction performance can be close to native. Microsecond VM creation is notable given the typical instance creation overhead in this case. A emulator VM for a sandbox typically will just be a software CPU emulator with some level of OS emulation. The instance creation overhead is setting up a data structure and issuing a function call to "run" the CPU emulator. The instruction performance is generally much slower than executing native code. Microsecond VM creation is not very notable in this case. If your running a long running process the hypervisor approach is usually superior. If your running a very short lived process (for instance a VM per http request), the emulator approach may work better. There is also the container approach like Docker which is somewhat in between in overhead and can run at native speed on bare metal. The OS virtualization GVisor approach of capturing syscalls.
ilaksh 2y ago
Is this better than Firecracker? I was thinking about using that but it needs nested virtualization and the servers that support that aren't as good of a value. Anyone know a good option for nested virtualization that is inexpensive? Hetzner is really cheap but not sure about the cost effectiveness for the dedicated servers. Actually I think what I saw was that I couldn't get the one I wanted in a US datacenter.
- fulafel 2y ago
  The cited startup time is better than Firecracker but it's not a better tool than Firecracker (for a start only claims to be a PoC, "contains only the necessary parts for realistic benchmarking"). Looks like it's based on RISC-V emulation so the approach seems unlikely to get performance comparable to Firecracker. You might look into gVisor if you're running containerized workloads on a host without virt support (such as a vm without nested virtualization support).
- fwsgonzo 2y ago
  Absolutely not! It's better to use something backed by KVM, so that you can use all the features of the CPU. This is just a proof-of-concept that was fun to make.
- nolist_policy 2y ago
  You could also use gVisor which doesn't need nested virtualization.
rollcat 2y ago
Tangential, but let's say I want to build a multiplayer game, where players (untrusted users) are allowed to run arbitrary code in some kind of a VM. I've so far established that: - The VM has to be built in a safe and performant language (like Rust, Go, or - if careful - modern C++), and available as a library to integrate with the rest of the game. However I don't trust myself to write safe C/C++ (the game is being prototyped in LÖVE/Lua). - Each VM instance needs a tight execution/instruction budget, to avoid stalling the server's main update loop; e.g. a timer/virtual "hardware interrupt", or simply counting cycles, or even something modelled after eBPF. The total number of VM instances running in a single game would also need to be limited somehow (e.g. making a key component scarce and/or non-renewable, or dividing the total instruction budget across all VMs, or requiring a player to be present in a nearby world chunk). Use cases are something like redstone in Minecraft: curious and technically-inclined players could build contraptions, like auto-farming crops, pranks/traps, defences, fancy gates/moats, etc. Not the core of the gameplay, but rather one aspect of it, for the curious to explore, learn, have fun with. There are many off-the-shelf VMs that do RISC-V or similar ISAs, and I'm considering picking one of those, but wondering if a RISC instruction set isn't too low-level for such a thing. On the other hand, it would be nice if the knowledge would be directly transferrable to the real world. Anyone tried to build something similar and can share their experience?
- whytevuhuni 2y ago
  Aren't you describing the ideal scenario for WASM here? Some runtimes like wasmtime have a concept of fuel [1], which can limit the execution time. One of the Rust-based Lua VMs, piccolo [2], was also made specifically with the ability to be interrupted in the middle of anything, by being stackless. [1] https://docs.wasmtime.dev/api/wasmtime/struct.Store.html#method.set_fuel https://docs.wasmtime.dev/api/wasmtime/struct.Store.html#met... [2] https://news.ycombinator.com/item?id=40239029 https://news.ycombinator.com/item?id=40239029 (see "Fuel" on its article)
  - rollcat 2y ago
    Actually Piccolo sounds like the ideal solution here, if I were to stick to LÖVE/Lua. It could be weird/confusing to have completely different VMs for the in-game contraptions vs the game engine itself (LuaJIT), but I think the narrow gap could attract players to become modders/contributors. Thanks for the suggestion!
owenthejumper 2y ago
Reminds me a bit of Cloudflare's isolates, but the title is super confusing
elmigranto 2y ago
> This project […] contains only the necessary parts for realistic benchmarking > The test program is a simple […] return string I understand how this is required to measure the effects of sandboxing in isolation. And the result is impressive. In what ways would you expect performance to be affected when workloads are more realistic as well?
- fwsgonzo 2y ago
  I have a bit of experience in this, and adding monitoring, logging and observability doesn't really affect it compared to the non-sandboxing path: All of those things should already be happening. There should already be logging and statistics gathering as part of the larger service. libriscv in interpreter mode is fast compared to other interpreters, but not near native performance. As I wrote earlier in the thread using something backed by KVM is what I would do if I were architecting a solution for someone. Eg. my TinyKVM KVM-based userspace emulator would fit the bill.
childintime 2y ago
This competes with WASM Serverless, therefore something like Fermyon Spin, which is built on top of it (https://www.fermyon.com/serverless-guide/speed-and-execution-time https://www.fermyon.com/serverless-guide/speed-and-execution...). Wake up a RISC-V emulator on a http request in 1µs, do your thing and exit. Then gone is the RISC-V VM. WASM takes a millisecond or more to spin up, as it is bytecode. Is there any cloud that provides RISC-V VM's, coupled with SQLite access for persistence?

84 points by fwsgonzo 2y ago

kjok 2y ago

What are the use cases?

selcuka 2y ago
Explained in the first and second paragraphs of the README: > Multi-tenancy allows one server to be safely shared among many users, each of which cannot access each others or negatively affect the HTTP service. > Specialized sandboxes are instantiated for each request and immediately destroyed after the request, all within a single microsecond.
- goodpoint 2y ago
  Some more practical examples would be nice, including security impact

pyaamb 2y ago

Whatever happened to Unikernels?

lgas 2y ago
Nothing.
yla92 2y ago
Still around. Recently, Prisma launched their managed Postgres offering built on top of Unikernels https://www.prisma.io/blog/announcing-prisma-postgres-early-access https://www.prisma.io/blog/announcing-prisma-postgres-early-...
goodpoint 2y ago
They are still in the same tiny niche

mdaniel 2y ago

This GitHub org was cited several times in the recent "source available" annoucement for MoonBit, if one wishes to see more "in the wild" usage https://news.ycombinator.com/item?id=42450274 https://news.ycombinator.com/item?id=42450274

thornewolf 2y ago

to check my understanding on what this is offering, I could build something on top of this that offers remote code execution for people without needing to worry about my system being compromised? or other people's processes interacting with one another, but the VM will still be able to make web requests itself?

fwsgonzo 2y ago
The VM could only make requests if you add a system call specifically for that purpose. Eg. if you add something like sys_request that takes a URL as argument and some input and output arguments. It's not that the VM couldn't open a socket and handle connections by itself, but it's all closed down. I think when it comes to being integrated into a web server, you ideally want to use the web servers pipeline to make requests, so that you can benefit from the same logging and observability that Drogon (or other solutions) already have. If you allow the VM to make requests on its own, you just have the IP and an open socket. Not much to debug with if (let's say) you have a customer asking about something.

jumploops 2y ago

What do VMs mean in this context? I did a pass of the codebase and it seems they’re just forking processes? It’s unclear to me where the safety guarantees come from (compared to using e.g. KVM). Edit: it appears the safety guarantees come from libriscv[0]. As far as I can tell, these sandboxes are essentially RISC-V programs running in an isolated context (“machine”) where all the Linux syscalls are emulated and thus “safe.” Still curious what potential attack vectors may exist? [0] https://github.com/libriscv/libriscv/tree/dfb7c85d01f01cb38fb34b529894a2baaa902d04 https://github.com/libriscv/libriscv/tree/dfb7c85d01f01cb38f...

RainyDayTmrw 2y ago
See also: Google's GVisor [1], which was used in Google App Engine. [1]: https://en.wikipedia.org/wiki/GVisor https://en.wikipedia.org/wiki/GVisor
fwsgonzo 2y ago
Well, the boundary between the host and the guest, the system call API, is always going to be the biggest vector of attacks no matter what the solution used is. But, if you find a problem and fix it, you're back to being safe again, unlike if you don't have any sandboxing at all. You can also put the whole solution in a jail, which is very common nowadays.
- jumploops 2y ago
  Can you expand on “put the whole solution in a jail”?
  - fwsgonzo 2y ago
    FireCracker has a jailer: https://github.com/firecracker-microvm/firecracker/blob/main/docs/jailer.md https://github.com/firecracker-microvm/firecracker/blob/main...
    - jumploops 2y ago
      Ah, this is helpful, thanks!

mattclarkdotnet 2y ago

Having scanned the codebase, I think this is about quickly and safely launching and managing risc-v binaries as sandboxed processes? Which is useful, but has nothing to do with virtual machines in the usual sense of there being a hypervisor with hardware support for isolation.

EgoIncarnate 2y ago

The use of the term VM without further qualification in the title is unfortunate. Emulated VM would have been nicer to avoid confusion with hypervisor style virtual machines. Staring emphermial hypervisor VMs quickly is more noteworthy (since they are often slow to start) than an emulator VM where it's expected to be fast since it's usually not much more than setting up a datastructure and executing a call to an interpreter. I clicked hoping for the former, only to find out the project is the latter.

binary132 2y ago
What exactly is the difference? Are you talking about hardware virtualization via e.g. intel vt-x? Do you mean virtualizing hardware subsystems with drivers instead of forwarding syscalls directly? Running a kernel? Maybe I’m not seeing how those things are fundamentally different than “setting up a datastructure and calling an interpreter”.
- EgoIncarnate 2y ago
  A hypervisor VM is running native code via hardware virtualization. An emulator VM is running an interpreter and/or JIT generally using a different instruction set. A hypervisor VM typically requires more extensive setup involving hardware configuration and is usually used for running existing native code, so it often means emulating a real machine, including OS and sometimes even firmware/BIOS. There are "lightweight" environments like Firecracker, but the overhead of creating an instance is still heaver than the overhead of a function call. The instance creation overhead is high, but the instruction performance can be close to native. Microsecond VM creation is notable given the typical instance creation overhead in this case. A emulator VM for a sandbox typically will just be a software CPU emulator with some level of OS emulation. The instance creation overhead is setting up a data structure and issuing a function call to "run" the CPU emulator. The instruction performance is generally much slower than executing native code. Microsecond VM creation is not very notable in this case. If your running a long running process the hypervisor approach is usually superior. If your running a very short lived process (for instance a VM per http request), the emulator approach may work better. There is also the container approach like Docker which is somewhat in between in overhead and can run at native speed on bare metal. The OS virtualization GVisor approach of capturing syscalls.

ilaksh 2y ago

Is this better than Firecracker? I was thinking about using that but it needs nested virtualization and the servers that support that aren't as good of a value. Anyone know a good option for nested virtualization that is inexpensive? Hetzner is really cheap but not sure about the cost effectiveness for the dedicated servers. Actually I think what I saw was that I couldn't get the one I wanted in a US datacenter.

fulafel 2y ago
The cited startup time is better than Firecracker but it's not a better tool than Firecracker (for a start only claims to be a PoC, "contains only the necessary parts for realistic benchmarking"). Looks like it's based on RISC-V emulation so the approach seems unlikely to get performance comparable to Firecracker. You might look into gVisor if you're running containerized workloads on a host without virt support (such as a vm without nested virtualization support).
fwsgonzo 2y ago
Absolutely not! It's better to use something backed by KVM, so that you can use all the features of the CPU. This is just a proof-of-concept that was fun to make.
nolist_policy 2y ago
You could also use gVisor which doesn't need nested virtualization.

rollcat 2y ago

Tangential, but let's say I want to build a multiplayer game, where players (untrusted users) are allowed to run arbitrary code in some kind of a VM. I've so far established that: - The VM has to be built in a safe and performant language (like Rust, Go, or - if careful - modern C++), and available as a library to integrate with the rest of the game. However I don't trust myself to write safe C/C++ (the game is being prototyped in LÖVE/Lua). - Each VM instance needs a tight execution/instruction budget, to avoid stalling the server's main update loop; e.g. a timer/virtual "hardware interrupt", or simply counting cycles, or even something modelled after eBPF. The total number of VM instances running in a single game would also need to be limited somehow (e.g. making a key component scarce and/or non-renewable, or dividing the total instruction budget across all VMs, or requiring a player to be present in a nearby world chunk). Use cases are something like redstone in Minecraft: curious and technically-inclined players could build contraptions, like auto-farming crops, pranks/traps, defences, fancy gates/moats, etc. Not the core of the gameplay, but rather one aspect of it, for the curious to explore, learn, have fun with. There are many off-the-shelf VMs that do RISC-V or similar ISAs, and I'm considering picking one of those, but wondering if a RISC instruction set isn't too low-level for such a thing. On the other hand, it would be nice if the knowledge would be directly transferrable to the real world. Anyone tried to build something similar and can share their experience?

whytevuhuni 2y ago
Aren't you describing the ideal scenario for WASM here? Some runtimes like wasmtime have a concept of fuel [1], which can limit the execution time. One of the Rust-based Lua VMs, piccolo [2], was also made specifically with the ability to be interrupted in the middle of anything, by being stackless. [1] https://docs.wasmtime.dev/api/wasmtime/struct.Store.html#method.set_fuel https://docs.wasmtime.dev/api/wasmtime/struct.Store.html#met... [2] https://news.ycombinator.com/item?id=40239029 https://news.ycombinator.com/item?id=40239029 (see "Fuel" on its article)
- rollcat 2y ago
  Actually Piccolo sounds like the ideal solution here, if I were to stick to LÖVE/Lua. It could be weird/confusing to have completely different VMs for the in-game contraptions vs the game engine itself (LuaJIT), but I think the narrow gap could attract players to become modders/contributors. Thanks for the suggestion!

owenthejumper 2y ago

Reminds me a bit of Cloudflare's isolates, but the title is super confusing

elmigranto 2y ago

> This project […] contains only the necessary parts for realistic benchmarking > The test program is a simple […] return string I understand how this is required to measure the effects of sandboxing in isolation. And the result is impressive. In what ways would you expect performance to be affected when workloads are more realistic as well?

fwsgonzo 2y ago
I have a bit of experience in this, and adding monitoring, logging and observability doesn't really affect it compared to the non-sandboxing path: All of those things should already be happening. There should already be logging and statistics gathering as part of the larger service. libriscv in interpreter mode is fast compared to other interpreters, but not near native performance. As I wrote earlier in the thread using something backed by KVM is what I would do if I were architecting a solution for someone. Eg. my TinyKVM KVM-based userspace emulator would fit the bill.

childintime 2y ago

This competes with WASM Serverless, therefore something like Fermyon Spin, which is built on top of it (https://www.fermyon.com/serverless-guide/speed-and-execution-time https://www.fermyon.com/serverless-guide/speed-and-execution...). Wake up a RISC-V emulator on a http request in 1µs, do your thing and exit. Then gone is the RISC-V VM. WASM takes a millisecond or more to spin up, as it is bytecode. Is there any cloud that provides RISC-V VM's, coupled with SQLite access for persistence?