3 ms·

> This is the biggest deal IMHO. FaaS in CDN edge DCs are remarkably fast and affordable. If you can respond to an API call faster than a request can get throug

by Zababa 5y ago

> This is the biggest deal IMHO. FaaS in CDN edge DCs are remarkably fast and affordable. If you can respond to an API call faster than a request can get through to a load balancer, you're providing quite a UX improvement regardless of cost. How much faster will this be? I think even for servers on the other side of the planet, the max ping will be something like 500ms. I think most pings in the same continent will be something like 100-200ms. On the other hand, a Node hello world was 164ms. That means that compared to just being in the same continent, you don't have much of an advantage. Python is ~30ms for a hello world, That's better. On the other hand, the "competition" is marked as 250ms for a cold start. That's a lot, and I doubt you get the benefits of being close to the user at this point. There may be some things I missed, I don't know much about that space.

reilly3000 5y ago
I can really only speak to CloudFlare workers in latency sensitive, user-facing settings, but from my experience their cold start times are insanely low, like 5-10ms. They use V8 Isolates (uberjar for js?) running WASM bytecode, so there is very little runtime overhead. So in that case, there is an actual benefit that being a short hop away, but otherwise I think you're right. I've also used some AWS Lambdas and GCP Cloud Functions in background data processing settings where there are under 1M invocations/month, so its cheaper than running a tiny instance 24/7. In that setting latency doesn't really matter as much, but I was seeing cold starts ~2 seconds and warm start invocations on my node and python code responding within 30-60ms in most cases. Once the container is up and running its reused for follow-on invocations, but eventually(and opaquely) times out and the next invocation will be cold. People who are using serverless runtimes have historically done hacky things to ensure they always have a running container like invoking the function with a cron job. At that point it seems a bit silly not to just run a small vm / container.
- Zababa 5y ago
  Thanks for the info. 5-10ms seems great, and a real plus for user experience. As you said, cold starts seem to be a killer, especially if you're trying to optimize the slowest requests.

by Zababa 5y ago

reilly3000 5y ago

I can really only speak to CloudFlare workers in latency sensitive, user-facing settings, but from my experience their cold start times are insanely low, like 5-10ms. They use V8 Isolates (uberjar for js?) running WASM bytecode, so there is very little runtime overhead. So in that case, there is an actual benefit that being a short hop away, but otherwise I think you're right. I've also used some AWS Lambdas and GCP Cloud Functions in background data processing settings where there are under 1M invocations/month, so its cheaper than running a tiny instance 24/7. In that setting latency doesn't really matter as much, but I was seeing cold starts ~2 seconds and warm start invocations on my node and python code responding within 30-60ms in most cases. Once the container is up and running its reused for follow-on invocations, but eventually(and opaquely) times out and the next invocation will be cold. People who are using serverless runtimes have historically done hacky things to ensure they always have a running container like invoking the function with a cron job. At that point it seems a bit silly not to just run a small vm / container.

Zababa 5y ago
Thanks for the info. 5-10ms seems great, and a real plus for user experience. As you said, cold starts seem to be a killer, especially if you're trying to optimize the slowest requests.