7 ms·

It's definitely impressive, kinda like an own serverless implementation. However, is there a point in shutting down the VM right away, other than being a cool

by htgb 9y ago

It's definitely impressive, kinda like an own serverless implementation. However, is there a point in shutting down the VM right away, other than being a cool demo? It seems to me that you could quite easily keep it around if there are more requests queued, and quickly spawn more workers if the queue gets too long. So a regular worker pool, just that the number of workers can be adjusted relatively quickly.

regularfry 9y ago
The point is that you don't need the added complexity of a worker pool. It's another thing to break, so why bother?
- michaelmior 9y ago
  Even 250ms latency to boot is significant. Especially if you're processing thousands of requests per second.
  - NegativeLatency 9y ago
    Or if you are processing more than 4 per core per second.
- imtringued 9y ago
  Threads take milliseconds to start yet we still use thread pools.
vidarh 9y ago
Shutting them down right away achieves a number of things, such as significantly better isolation between requests, and more protection against resource leaks. Whether that'll be worth the cost remains to be seen - 250ms is still far too long for most uses, but if it can come down enough, it's attractive.
- rwmj 9y ago
  I couldn't get to the site, but assuming this is running Linux I wrote a paper about how fast you could boot a Linux minimal userspace on QEMU a couple of years ago[1]. In short if you're willing to heavily customize the kernel, qemu and userspace then you can get it down to 100-150ms on i7-type hardware, but probably not much lower. Using a distro kernel, distro qemu and so on, you're stuck with 400-600ms (which is fine for certain uses, eg better security for containers like Kata Containers is doing). Also worth noting I wrote a bunch of benchmarking tools to help optimize this case[2]. [1] http://oirase.annexia.org/tmp/paper.pdf http://oirase.annexia.org/tmp/paper.pdf [2] https://github.com/libguestfs/libguestfs/tree/master/utils https://github.com/libguestfs/libguestfs/tree/master/utils
  - deleted 9y ago
    [deleted]
  - vidarh 9y ago
    Thanks for the links. I've looked at Clear Containers/Hyper/Kata before, and it's absolutely looking good. It's very interesting to see your work on going into details on it, though. I really hope this can be pushed further down to the point where it gets viable to for more use cases. Of course, we can always use something lighter than Linux, but being able to start with Linux and pare down is potentially so much easier for many types of applications.
pjc50 9y ago
> own serverless implementation Isn't this just cgi-bin, right down to the long startup time and lack of persistence?
- htgb 9y ago
  Is it? I'm under the impression that cgi-bin is a way to serve requests from the local computer, while Zerg is a way to spawn _other_ server instances to handle the request (and in Erlang, specifically).
  - pjc50 9y ago
    Yes, but you could let a load balancer handle that for you, and you might not need as many computers to handle the requests if it didn't take 250ms to start.
    - sitkack 9y ago
      Booting a VM should be "as complex" as fork. And the memory image of the booted OS should be identical on each load before the end user specific data is loaded. "Prefork" these VMs by spraying that image over physical memory in advance, jump to them on a request. memcopy a new VM over them on exit. VM spawn rate should be the memcopy rate, if the MMU is powerful enough, should only be the amount of memory mutated during a request.
- jxf 9y ago
  It's similar in intention, but "serverless is just `cgi-bin`" misses a lot of nuance. Zerg and cgi-bin are solving different problems and with different implementations. It's extremely difficult, if not impossible in some environments, to get the same kind of isolation with cgi-bin that you do out of a VM.
  - catern 9y ago
    You gain on isolation with Zerg (and other "VM per request" things) but you lose on everything else compared to CGI with processes. But you can reach comparable levels of isolation with processes: just use Linux namespaces.
hanbura 9y ago
I have written a few web services in the past where this kind of isolation would have been great. Services doing file conversion of untrusted input with software that isn't hardened for that case (in my case converting PDF to png and postscript).

by htgb 9y ago

regularfry 9y ago

The point is that you don't need the added complexity of a worker pool. It's another thing to break, so why bother?

michaelmior 9y ago
Even 250ms latency to boot is significant. Especially if you're processing thousands of requests per second.
- NegativeLatency 9y ago
  Or if you are processing more than 4 per core per second.
imtringued 9y ago
Threads take milliseconds to start yet we still use thread pools.

vidarh 9y ago

Shutting them down right away achieves a number of things, such as significantly better isolation between requests, and more protection against resource leaks. Whether that'll be worth the cost remains to be seen - 250ms is still far too long for most uses, but if it can come down enough, it's attractive.

rwmj 9y ago
I couldn't get to the site, but assuming this is running Linux I wrote a paper about how fast you could boot a Linux minimal userspace on QEMU a couple of years ago[1]. In short if you're willing to heavily customize the kernel, qemu and userspace then you can get it down to 100-150ms on i7-type hardware, but probably not much lower. Using a distro kernel, distro qemu and so on, you're stuck with 400-600ms (which is fine for certain uses, eg better security for containers like Kata Containers is doing). Also worth noting I wrote a bunch of benchmarking tools to help optimize this case[2]. [1] http://oirase.annexia.org/tmp/paper.pdf http://oirase.annexia.org/tmp/paper.pdf [2] https://github.com/libguestfs/libguestfs/tree/master/utils https://github.com/libguestfs/libguestfs/tree/master/utils
- deleted 9y ago
  [deleted]
- vidarh 9y ago
  Thanks for the links. I've looked at Clear Containers/Hyper/Kata before, and it's absolutely looking good. It's very interesting to see your work on going into details on it, though. I really hope this can be pushed further down to the point where it gets viable to for more use cases. Of course, we can always use something lighter than Linux, but being able to start with Linux and pare down is potentially so much easier for many types of applications.

pjc50 9y ago

> own serverless implementation Isn't this just cgi-bin, right down to the long startup time and lack of persistence?

htgb 9y ago
Is it? I'm under the impression that cgi-bin is a way to serve requests from the local computer, while Zerg is a way to spawn _other_ server instances to handle the request (and in Erlang, specifically).
- pjc50 9y ago
  Yes, but you could let a load balancer handle that for you, and you might not need as many computers to handle the requests if it didn't take 250ms to start.
  - sitkack 9y ago
    Booting a VM should be "as complex" as fork. And the memory image of the booted OS should be identical on each load before the end user specific data is loaded. "Prefork" these VMs by spraying that image over physical memory in advance, jump to them on a request. memcopy a new VM over them on exit. VM spawn rate should be the memcopy rate, if the MMU is powerful enough, should only be the amount of memory mutated during a request.
jxf 9y ago
It's similar in intention, but "serverless is just `cgi-bin`" misses a lot of nuance. Zerg and cgi-bin are solving different problems and with different implementations. It's extremely difficult, if not impossible in some environments, to get the same kind of isolation with cgi-bin that you do out of a VM.
- catern 9y ago
  You gain on isolation with Zerg (and other "VM per request" things) but you lose on everything else compared to CGI with processes. But you can reach comparable levels of isolation with processes: just use Linux namespaces.

hanbura 9y ago

I have written a few web services in the past where this kind of isolation would have been great. Services doing file conversion of untrusted input with software that isn't hardened for that case (in my case converting PDF to png and postscript).