3 ms·

I think Lambda should do this. Lambda seems like its a more managed version of the concepts in inetd + CGI: Like inetd, Lambda will start a server, if needed, t

by jitl 9y ago

I think Lambda should do this. Lambda seems like its a more managed version of the concepts in inetd + CGI: Like inetd, Lambda will start a server, if needed, to handle a new connection, and like CGI, Lambda passes request data to user application code in a standardized way. You could implement a Lambda-like service using inetd + CGI to run functions in Docker containers, and then destroy the container once you get the result. Spawning a process in a container is expensive. Not only do you need to wait for the container to init and mount its filesystems, but you then need to wait for the inner application to boot on every request, on top of that. So, allow containers to linger with applications booted, like Lambda does to save on those starts. Since we aren't launching a new process for every call, we can't use CGI with its environment variables, etc, so we'll have to describe a new protocol for calling your functions, like... HTTP! The interface could be POST /call with JSON params. I'm not sure how this model would differ semantically from the fully-managed, in-process system that Lambda uses. The management system can always kill containers that go over their memory limits or seem to be doing suspicious amounts of system calls when they're not executing a function. AWS can still go about providing in-process libraries for "supported" languages that implement these protocols, but if they also describe these protocols, then the community can go ahead and develop libraries for other languages that could eventually be adopted and officially supported. In fact, the community is already producing libraries that use AWS's supported languages to allow hosting lambdas in other languages, via a CGI-like model! Just look at https://github.com/apex/apex https://github.com/apex/apex - a NodeJS shim will spawn a command in the language of your choice, and then return the result back to Lambda.

lobster_johnson 9y ago
Why would a container be slow to create? It's effectively a fork with namespacing options. The overlay file system is just a product of one or more layers that are already computed; there's not much complexity to its mounting compared to something like ext4 or NFS. I haven't timed it, but containers seem damn fast to start with Docker and Kubernetes.
- TheDong 9y ago
  > I haven't timed it, but containers seem damn fast to start with Docker and Kubernetes. You should time it then, it's incredibly slow these days $ time docker run busybox /bin/true .388s total $ time docker run --net=none busybox /bin/true .310s total The above times, of over 300ms, are with docker 17.06 and the overlay2 driver on a powerful computer with an ssd. It's horrifically slow because the entire docker codebase is disgustingly bloated. It has to fork something like 10 processes and synchronize between them (for everything from network initialization to runc hooks). It has to make ipc calls from docker-cli to dockerd to containerd and back again, allocating and freeing large chunks of memory for grpc along the way. It's incredibly slow. On the other hand, just forking and running something with namespacing options is effectively instant: $ dir=$(mktemp -d) $ docker run --name bbexport busybox /bin/true $ cd $dir $ docker export bbexport | tar x $ time nsenter --root=$dir /bin/true 0.000s total This is the reason they don't let you use arbitrary docker containers; docker containers are a ton more than a fork and some options. They're a bloated moving target. What lambda does allows them to optmize much more. In fact, lambda doesn't even have to fork a process sometimes because they can keep the e.g. javascript runtime running and then just add a new request into the existing process directly. They can freeze and thaw environments from known good states for request processing. But most importantly, by not being compatible in any way with docker, they don't have to deal with the bloated mess that is docker and the ecosystem around it. They don't have to load 3 daemons measuring 50MB into memory to run 100kb of javascript (which they can currently run more quickly than docker can even start the simpleist container)
- jitl 9y ago
  Even though a container starts "damn fast" in Docker, you're still adding significant latency to each call of your function if it requires a new container. $ time docker run alpine:3.6 sh -c exit docker run alpine:3.6 sh -c exit 0.02s user 0.02s system 3% cpu 1.191 total $ time sh -c exit sh -c exit 0.00s user 0.01s system 94% cpu 0.007 total Then add to that the cost of loading your function's code into memory. For an interpreted language, that is probably going to take a while as the runtime does a bunch of system calls and parses text files. Even for a compiled, statically-linked language like Go, the executable (a random binary I picked was 50MB) needs to be copied into RAM. For this reason, Lambda tries to keep your containers/function-wrapper-applications booted up in RAM: https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/ https://aws.amazon.com/blogs/compute/container-reuse-in-lamb...

by jitl 9y ago

lobster_johnson 9y ago

Why would a container be slow to create? It's effectively a fork with namespacing options. The overlay file system is just a product of one or more layers that are already computed; there's not much complexity to its mounting compared to something like ext4 or NFS. I haven't timed it, but containers seem damn fast to start with Docker and Kubernetes.

TheDong 9y ago
> I haven't timed it, but containers seem damn fast to start with Docker and Kubernetes. You should time it then, it's incredibly slow these days $ time docker run busybox /bin/true .388s total $ time docker run --net=none busybox /bin/true .310s total The above times, of over 300ms, are with docker 17.06 and the overlay2 driver on a powerful computer with an ssd. It's horrifically slow because the entire docker codebase is disgustingly bloated. It has to fork something like 10 processes and synchronize between them (for everything from network initialization to runc hooks). It has to make ipc calls from docker-cli to dockerd to containerd and back again, allocating and freeing large chunks of memory for grpc along the way. It's incredibly slow. On the other hand, just forking and running something with namespacing options is effectively instant: $ dir=$(mktemp -d) $ docker run --name bbexport busybox /bin/true $ cd $dir $ docker export bbexport | tar x $ time nsenter --root=$dir /bin/true 0.000s total This is the reason they don't let you use arbitrary docker containers; docker containers are a ton more than a fork and some options. They're a bloated moving target. What lambda does allows them to optmize much more. In fact, lambda doesn't even have to fork a process sometimes because they can keep the e.g. javascript runtime running and then just add a new request into the existing process directly. They can freeze and thaw environments from known good states for request processing. But most importantly, by not being compatible in any way with docker, they don't have to deal with the bloated mess that is docker and the ecosystem around it. They don't have to load 3 daemons measuring 50MB into memory to run 100kb of javascript (which they can currently run more quickly than docker can even start the simpleist container)
jitl 9y ago
Even though a container starts "damn fast" in Docker, you're still adding significant latency to each call of your function if it requires a new container. $ time docker run alpine:3.6 sh -c exit docker run alpine:3.6 sh -c exit 0.02s user 0.02s system 3% cpu 1.191 total $ time sh -c exit sh -c exit 0.00s user 0.01s system 94% cpu 0.007 total Then add to that the cost of loading your function's code into memory. For an interpreted language, that is probably going to take a while as the runtime does a bunch of system calls and parses text files. Even for a compiled, statically-linked language like Go, the executable (a random binary I picked was 50MB) needs to be copied into RAM. For this reason, Lambda tries to keep your containers/function-wrapper-applications booted up in RAM: https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/ https://aws.amazon.com/blogs/compute/container-reuse-in-lamb...