3 ms·

I'm curious about latency, cold and warm, using docker. I have a dockerized number cruncher and it's a breeze to maintain, and I'm thinking of moving everything

I'm curious about latency, cold and warm, using docker. I have a dockerized number cruncher and it's a breeze to maintain, and I'm thinking of moving everything over. What's your experience?

vermilingua 3y ago
My experience with rust coldstarts was very good, my ECS backed lambdas coldstart and return a response in <40ms.
- jvanderbot 3y ago
  Yeah at this point vercel and AWS and basically everyone support serverless docker. It's probably dumb to do anything else.
  - maerF0x0 3y ago
    There are lots of kinds of containerization too btw, if i'm not mistaken AWS has a lot of investment in Firecracker too https://firecracker-microvm.github.io/ https://firecracker-microvm.github.io/
arrakeenrevived 3y ago
My understanding is that cold starts on containerized Lambdas is actually better than non-containerized for some workloads, because using containers allows Lambda to do better caching of the code, as well as lazy-loading. YMMV of course based on exactly what image you use (eg if you're not using a common base, like Ubuntu or Amazon Linux, you won't get as much benefit from the caching) and how much custom code you have (like hundreds of MBs worth). There's a very interesting blog post about it here, as well an an accompanying whitepaper: https://brooker.co.za/blog/2023/05/23/snapshot-loading.html https://brooker.co.za/blog/2023/05/23/snapshot-loading.html
maerF0x0 3y ago
I never had a case where cold starts mattered because either 1) it was the kind of service where cold starts intrinsically didnt matter, or 2) we generally had > 1 req/15mins meaning we always had something warm. 3) Also you can pay for provisioned capacity[1] if the cold start thing makes it worth the money, though also just look into fargate[2] if that's the case. [1]: https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html https://docs.aws.amazon.com/lambda/latest/dg/configuration-c... [2]: https://aws.amazon.com/fargate/ https://aws.amazon.com/fargate/
tracker1 3y ago
Docker is a bit more cold start time over native (zipped). That said, rust is so much faster than the scripted languages it's still much faster than what most are doing.

I'm curious about latency, cold and warm, using docker. I have a dockerized number cruncher and it's a breeze to maintain, and I'm thinking of moving everything over. What's your experience?

vermilingua 3y ago

My experience with rust coldstarts was very good, my ECS backed lambdas coldstart and return a response in <40ms.

jvanderbot 3y ago
Yeah at this point vercel and AWS and basically everyone support serverless docker. It's probably dumb to do anything else.
- maerF0x0 3y ago
  There are lots of kinds of containerization too btw, if i'm not mistaken AWS has a lot of investment in Firecracker too https://firecracker-microvm.github.io/ https://firecracker-microvm.github.io/

arrakeenrevived 3y ago

My understanding is that cold starts on containerized Lambdas is actually better than non-containerized for some workloads, because using containers allows Lambda to do better caching of the code, as well as lazy-loading. YMMV of course based on exactly what image you use (eg if you're not using a common base, like Ubuntu or Amazon Linux, you won't get as much benefit from the caching) and how much custom code you have (like hundreds of MBs worth). There's a very interesting blog post about it here, as well an an accompanying whitepaper: https://brooker.co.za/blog/2023/05/23/snapshot-loading.html https://brooker.co.za/blog/2023/05/23/snapshot-loading.html

maerF0x0 3y ago

I never had a case where cold starts mattered because either 1) it was the kind of service where cold starts intrinsically didnt matter, or 2) we generally had > 1 req/15mins meaning we always had something warm. 3) Also you can pay for provisioned capacity[1] if the cold start thing makes it worth the money, though also just look into fargate[2] if that's the case. [1]: https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html https://docs.aws.amazon.com/lambda/latest/dg/configuration-c... [2]: https://aws.amazon.com/fargate/ https://aws.amazon.com/fargate/

tracker1 3y ago

Docker is a bit more cold start time over native (zipped). That said, rust is so much faster than the scripted languages it's still much faster than what most are doing.