4 ms·

This plus because you can dockerize and run on lambda, essentially you can run most anything these days (most things i've encountered are reasonably easy to doc

This plus because you can dockerize and run on lambda, essentially you can run most anything these days (most things i've encountered are reasonably easy to dockerize, i'm sure there are exceptions, but in the main easy)

jvanderbot 3y ago
I'm curious about latency, cold and warm, using docker. I have a dockerized number cruncher and it's a breeze to maintain, and I'm thinking of moving everything over. What's your experience?
- vermilingua 3y ago
  My experience with rust coldstarts was very good, my ECS backed lambdas coldstart and return a response in <40ms.
  - jvanderbot 3y ago
    Yeah at this point vercel and AWS and basically everyone support serverless docker. It's probably dumb to do anything else.
    - maerF0x0 3y ago
      There are lots of kinds of containerization too btw, if i'm not mistaken AWS has a lot of investment in Firecracker too https://firecracker-microvm.github.io/ https://firecracker-microvm.github.io/
- arrakeenrevived 3y ago
  My understanding is that cold starts on containerized Lambdas is actually better than non-containerized for some workloads, because using containers allows Lambda to do better caching of the code, as well as lazy-loading. YMMV of course based on exactly what image you use (eg if you're not using a common base, like Ubuntu or Amazon Linux, you won't get as much benefit from the caching) and how much custom code you have (like hundreds of MBs worth). There's a very interesting blog post about it here, as well an an accompanying whitepaper: https://brooker.co.za/blog/2023/05/23/snapshot-loading.html https://brooker.co.za/blog/2023/05/23/snapshot-loading.html
- maerF0x0 3y ago
  I never had a case where cold starts mattered because either 1) it was the kind of service where cold starts intrinsically didnt matter, or 2) we generally had > 1 req/15mins meaning we always had something warm. 3) Also you can pay for provisioned capacity[1] if the cold start thing makes it worth the money, though also just look into fargate[2] if that's the case. [1]: https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html https://docs.aws.amazon.com/lambda/latest/dg/configuration-c... [2]: https://aws.amazon.com/fargate/ https://aws.amazon.com/fargate/
- tracker1 3y ago
  Docker is a bit more cold start time over native (zipped). That said, rust is so much faster than the scripted languages it's still much faster than what most are doing.

jvanderbot 3y ago

I'm curious about latency, cold and warm, using docker. I have a dockerized number cruncher and it's a breeze to maintain, and I'm thinking of moving everything over. What's your experience?

vermilingua 3y ago
My experience with rust coldstarts was very good, my ECS backed lambdas coldstart and return a response in <40ms.
- jvanderbot 3y ago
  Yeah at this point vercel and AWS and basically everyone support serverless docker. It's probably dumb to do anything else.
  - maerF0x0 3y ago
    There are lots of kinds of containerization too btw, if i'm not mistaken AWS has a lot of investment in Firecracker too https://firecracker-microvm.github.io/ https://firecracker-microvm.github.io/
arrakeenrevived 3y ago
My understanding is that cold starts on containerized Lambdas is actually better than non-containerized for some workloads, because using containers allows Lambda to do better caching of the code, as well as lazy-loading. YMMV of course based on exactly what image you use (eg if you're not using a common base, like Ubuntu or Amazon Linux, you won't get as much benefit from the caching) and how much custom code you have (like hundreds of MBs worth). There's a very interesting blog post about it here, as well an an accompanying whitepaper: https://brooker.co.za/blog/2023/05/23/snapshot-loading.html https://brooker.co.za/blog/2023/05/23/snapshot-loading.html
maerF0x0 3y ago
I never had a case where cold starts mattered because either 1) it was the kind of service where cold starts intrinsically didnt matter, or 2) we generally had > 1 req/15mins meaning we always had something warm. 3) Also you can pay for provisioned capacity[1] if the cold start thing makes it worth the money, though also just look into fargate[2] if that's the case. [1]: https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html https://docs.aws.amazon.com/lambda/latest/dg/configuration-c... [2]: https://aws.amazon.com/fargate/ https://aws.amazon.com/fargate/
tracker1 3y ago
Docker is a bit more cold start time over native (zipped). That said, rust is so much faster than the scripted languages it's still much faster than what most are doing.