4 ms·

No cold-starts means no overhead of starting a process to answer a request like most container-based serverless environments (without having to keep pre-warmed

No cold-starts means no overhead of starting a process to answer a request like most container-based serverless environments (without having to keep pre-warmed instances which kind of defeats the purpose) A couple references regarding cold-starts and performance in serverless environments. https://www.fastly.com/blog/lucet-performance-and-lifecycle https://www.fastly.com/blog/lucet-performance-and-lifecycle https://arxiv.org/abs/2010.07115 https://arxiv.org/abs/2010.07115

kllrnohj 4y ago
Are we so what's-old-is-new-again as to be re-inventing fast-cgi at this point? And why are we pretending this has anything to do with WASM instead of just how your API/service is designed?
- ridruejo 4y ago
  That’s basically the history of computing! There are some intrinsic advantages to using Wasm vs VMs or containers for certain scenarios, like serverless. That’s very similar to what’s going on with cloudflare workers and V8 isolates. It’s not one size fits all by any means, but it is certainly really good for many scenarios where containers are not
- laurencerowe 4y ago
  FastCGI reuses the same process for multiple requests. As I understand it wasmtime now supports very fast startup so you can use a new instance per request (avoiding the risk of inter-request bugs) with very low overhead (5 microseconds on their benchmark https://bytecodealliance.org/articles/wasmtime-10-performance https://bytecodealliance.org/articles/wasmtime-10-performanc....) With Firecracker I believe snapshot restore time is around 2-3ms. In my tests wasmtime ran about 50% the speed of native so depending on your workload it might still be faster for short running jobs where the startup time dominates. (Wasmer was maybe 80-90% of native speed but I don't know their startup times.)
  - kllrnohj 4y ago
    > As I understand it wasmtime now supports very fast startup WASM startup isn't going to be any faster than native code startup. It's going to be strictly worse if anything thanks to the JIT, although you can AOT that to native and then just restore parity with native code. Which just gets back to the speed of your startup depends on what your startup does.
    - laurencerowe 4y ago
      From what I understand, wasmtime's fast startup is conceptually similar to forking a process per instantiation, but much faster since it uses lazy initialization and has fewer operating system resources to setup. Some of those techniques can be applied to native code too, see "On-demand-fork: A Microsecond Fork for Memory-Intensive and Latency-Sensitive Applications" https://www.cs.purdue.edu/homes/pfonseca/papers/eurosys21-odf.pdf https://www.cs.purdue.edu/homes/pfonseca/papers/eurosys21-od... But I think wasmtime can always be faster to instantiate since the guarantees provided by the runtime allow it to safely reset and reuse instantiations: "We implemented an “instance allocator” in Wasmtime that makes use of this copy-on-write (CoW) technique for very fast instantiations. It also uses a Linux syscall known as madvise to quickly “reset” the page mappings back to the original read-only heap image, so we can reuse the same mappings over and over when the same Wasm program is re-instantiated many times. (One might imagine this would be the case in a server serving many requests, for example!)" https://bytecodealliance.org/articles/wasmtime-10-performance https://bytecodealliance.org/articles/wasmtime-10-performanc...
  - bombolo 4y ago
    > you can use a new instance per request I doubt this will ever be as fast
    - laurencerowe 4y ago
      What approach would be faster that provides similar isolation? It certainly seems a lot faster than fork and seemingly faster than the on-demand-fork I mentioned elsewhere.

kllrnohj 4y ago

Are we so what's-old-is-new-again as to be re-inventing fast-cgi at this point? And why are we pretending this has anything to do with WASM instead of just how your API/service is designed?

ridruejo 4y ago
That’s basically the history of computing! There are some intrinsic advantages to using Wasm vs VMs or containers for certain scenarios, like serverless. That’s very similar to what’s going on with cloudflare workers and V8 isolates. It’s not one size fits all by any means, but it is certainly really good for many scenarios where containers are not
laurencerowe 4y ago
FastCGI reuses the same process for multiple requests. As I understand it wasmtime now supports very fast startup so you can use a new instance per request (avoiding the risk of inter-request bugs) with very low overhead (5 microseconds on their benchmark https://bytecodealliance.org/articles/wasmtime-10-performance https://bytecodealliance.org/articles/wasmtime-10-performanc....) With Firecracker I believe snapshot restore time is around 2-3ms. In my tests wasmtime ran about 50% the speed of native so depending on your workload it might still be faster for short running jobs where the startup time dominates. (Wasmer was maybe 80-90% of native speed but I don't know their startup times.)
- kllrnohj 4y ago
  > As I understand it wasmtime now supports very fast startup WASM startup isn't going to be any faster than native code startup. It's going to be strictly worse if anything thanks to the JIT, although you can AOT that to native and then just restore parity with native code. Which just gets back to the speed of your startup depends on what your startup does.
  - laurencerowe 4y ago
    From what I understand, wasmtime's fast startup is conceptually similar to forking a process per instantiation, but much faster since it uses lazy initialization and has fewer operating system resources to setup. Some of those techniques can be applied to native code too, see "On-demand-fork: A Microsecond Fork for Memory-Intensive and Latency-Sensitive Applications" https://www.cs.purdue.edu/homes/pfonseca/papers/eurosys21-odf.pdf https://www.cs.purdue.edu/homes/pfonseca/papers/eurosys21-od... But I think wasmtime can always be faster to instantiate since the guarantees provided by the runtime allow it to safely reset and reuse instantiations: "We implemented an “instance allocator” in Wasmtime that makes use of this copy-on-write (CoW) technique for very fast instantiations. It also uses a Linux syscall known as madvise to quickly “reset” the page mappings back to the original read-only heap image, so we can reuse the same mappings over and over when the same Wasm program is re-instantiated many times. (One might imagine this would be the case in a server serving many requests, for example!)" https://bytecodealliance.org/articles/wasmtime-10-performance https://bytecodealliance.org/articles/wasmtime-10-performanc...
- bombolo 4y ago
  > you can use a new instance per request I doubt this will ever be as fast
  - laurencerowe 4y ago
    What approach would be faster that provides similar isolation? It certainly seems a lot faster than fork and seemingly faster than the on-demand-fork I mentioned elsewhere.