6 ms·

You’ll have to deal with lambda cold starts if you want it to be performant: > When the Lambda service receives a request to run a function via the Lambda API,

You’ll have to deal with lambda cold starts if you want it to be performant: > When the Lambda service receives a request to run a function via the Lambda API, the service first prepares an execution environment. During this step, the service downloads the code for the function, which is stored in an internal Amazon S3 bucket (or in Amazon Elastic Container Registry if the function uses container packaging). It then creates an environment with the memory, runtime, and configuration specified. Once complete, Lambda runs any initialization code outside of the event handler before finally running the handler code. https://aws.amazon.com/blogs/compute/operating-lambda-performance-optimization-part-1/ https://aws.amazon.com/blogs/compute/operating-lambda-perfor...

booi 4y ago
This is definitely an issue especially with infrequently accessed functions but I've seen cold start issues regardless. I assume some scaling events will cause cold starts (measured in seconds). There's a good argument to go with packaged code instead of containers if you can manage the development complication and versioning (cold starts measured in milliseconds).
- cebert 4y ago
  My team owns a Node 14 JS lambda application that is completely serverless. We’re focused on keeping our lambdas small with single responsibilities and leverage lambda layers for anything common across multiple lambdas. Cold starts are a concern, but is very negligible (< 100ms tops) and unnoticed by our client apps. We host all of our static web and JS assets via Cloud Front so they load quickly. If a user happened to visit our site when a lambda incurred a cold start, it’s not perceptible. We were much more worried initially about cold starts than it turned out we needed to be. Keeping lambdas small, picking a good language for the job, and leveraging lambda layers help minimize this a lot.
mjb 4y ago
It's not entirely accurate that Lambda pulls container images from ECR at start-up time. Here's me talking about what happens behind the scenes (which, in the real world, often makes things orders of magnitude faster than a full container pull): https://www.youtube.com/watch?v=A-7j0QlGwFk https://www.youtube.com/watch?v=A-7j0QlGwFk But your broader point is correct. Cold starts are a challenge, but they're one that the team is constantly working on and improving. You can also help reduce cold-start time by picking languages without heavy VMs (Go, Rust, etc), but reducing work done in 'static' code, and by minimizing the size of your container image. All those things will get less important over time, but they all can have a huge impact on cold-starts now. Another option is Lambda Provisioned concurrency, which allows you to pay a small amount to control how many sandboxes Lambda keeps warm on your behalf: https://docs.aws.amazon.com/lambda/latest/dg/provisioned-concurrency.html https://docs.aws.amazon.com/lambda/latest/dg/provisioned-con...
- afandian 4y ago
  Pardon the ignorance, but is the state of lambda containers considered to be single-threaded? Or can they serve requests in parallel? If I had a Spring Java (well, Kotlin) app that processes stuff off SQS (large startup time but potentially very high parallelism), would you recommend running ECS containers and scale them up based on SQS back-pressure? Or would you package them up as Lambdas with provisioned capacity? Throughput will be fairly consistent (never zero) and occasionally bursty.
  - catlifeonmars 4y ago
    Each container handles requests serially. This doesn’t preclude you from spawning multiple threads in Lambda to do background work though.
    - shortstuffsushi 4y ago
      Serially, but up to ten requests in a single batch > By default, Lambda polls up to 10 messages in your queue at once and sends that batch to your function. From https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html
  - dimitrios1 4y ago
    I would not use Spring, or Java for that matter, for lambdas, speaking from experience. "Lambda containers" is a bit of a misnomer, as you can have multiple instances of a function run on the same container, it's just that initial startup time once the container shuts down that is slow (which can be somewhat avoided by a "warming" function set to trigger on a cron). I would definitely go with containers if your intention is to use Spring. ECS containers can autoscale just the same as lambdas. There's some work being done to package Java code to run more efficiently in serverless computing environments, but IIRC, it's not there yet.
    - afandian 4y ago
      Thanks! I wasn't planning it, but can't hurt to ask. When I looked the Lambda API looked uncomplicated to implement (I saw an example somewhere) and it felt like you could just write a few controllers and gain the ability to run a subset of functionality in Lambda, especially if your app could be kept warm. (to your cron comment, I thought that the reserved capacity would mean the container would be forcibly kept warm?)
daenz 4y ago
They have a feature called "provisioned concurrency" where basically one "instance" of your lambda (or however many you want to configure) stays running warm, so that it can handle requests quickly. I know it defeats the conceptual purpose of serverless, but it's a nice workaround while cloud platforms work on mitigating the cold start problem.
- onphonenow 4y ago
  I've had some pretty good luck sliming things down as well. That's a win win usually for even non-lambda cases (trying things like docker-slim / switching stuff to go that needs a quick response). That said, the $2-5/month is fine as well for some cases.
- catlifeonmars 4y ago
  It’s nice to have that dial. Running lambda with provisioned concurrency is still a very managed experience: much different than running a container cluster.
- andrew_ 4y ago
  That'll also cost you $$$$ and takes any provisioned lambda out of the free tier. Also note that only your specified number of instances will stay warm, meaning if your lambda needs to scale up, you risk slow cold starts on additional instances outside of the number you provisioned. You could specify the number of reserved concurrent images (limiting the number that run) but that also costs money and will eat int your quota. Using containers for lambda is generally a bad idea for anything TypeScript/JavaScript that handles a realtime request - you just can't beat the speed of a single JavaScript file (compiled, in the case of TS). AWS CDK now ships with the NodeJSFunction as well, which makes generating those a breeze with ESBuild.
  - daenz 4y ago
    It cost me like $3 a month to get benefit from it. For what it's worth I don't use it for synchronous web requests.
sam0x17 4y ago
If cold starts are at all an issue for whatever use-case, you can just do a warming job like we do (in our case it's built into Ruby on Jets). We find invoking every 30 seconds is enough to never have a cold start. It's still quite cheap as well. The lambda portion of our bill (with tons of platform usage) is still incredibly low / low double digits. Just doing a warming job with no other usage falls well within free tier usage, I can confirm.

booi 4y ago

This is definitely an issue especially with infrequently accessed functions but I've seen cold start issues regardless. I assume some scaling events will cause cold starts (measured in seconds). There's a good argument to go with packaged code instead of containers if you can manage the development complication and versioning (cold starts measured in milliseconds).

cebert 4y ago
My team owns a Node 14 JS lambda application that is completely serverless. We’re focused on keeping our lambdas small with single responsibilities and leverage lambda layers for anything common across multiple lambdas. Cold starts are a concern, but is very negligible (< 100ms tops) and unnoticed by our client apps. We host all of our static web and JS assets via Cloud Front so they load quickly. If a user happened to visit our site when a lambda incurred a cold start, it’s not perceptible. We were much more worried initially about cold starts than it turned out we needed to be. Keeping lambdas small, picking a good language for the job, and leveraging lambda layers help minimize this a lot.

mjb 4y ago

It's not entirely accurate that Lambda pulls container images from ECR at start-up time. Here's me talking about what happens behind the scenes (which, in the real world, often makes things orders of magnitude faster than a full container pull): https://www.youtube.com/watch?v=A-7j0QlGwFk https://www.youtube.com/watch?v=A-7j0QlGwFk But your broader point is correct. Cold starts are a challenge, but they're one that the team is constantly working on and improving. You can also help reduce cold-start time by picking languages without heavy VMs (Go, Rust, etc), but reducing work done in 'static' code, and by minimizing the size of your container image. All those things will get less important over time, but they all can have a huge impact on cold-starts now. Another option is Lambda Provisioned concurrency, which allows you to pay a small amount to control how many sandboxes Lambda keeps warm on your behalf: https://docs.aws.amazon.com/lambda/latest/dg/provisioned-concurrency.html https://docs.aws.amazon.com/lambda/latest/dg/provisioned-con...

afandian 4y ago
Pardon the ignorance, but is the state of lambda containers considered to be single-threaded? Or can they serve requests in parallel? If I had a Spring Java (well, Kotlin) app that processes stuff off SQS (large startup time but potentially very high parallelism), would you recommend running ECS containers and scale them up based on SQS back-pressure? Or would you package them up as Lambdas with provisioned capacity? Throughput will be fairly consistent (never zero) and occasionally bursty.
- catlifeonmars 4y ago
  Each container handles requests serially. This doesn’t preclude you from spawning multiple threads in Lambda to do background work though.
  - shortstuffsushi 4y ago
    Serially, but up to ten requests in a single batch > By default, Lambda polls up to 10 messages in your queue at once and sends that batch to your function. From https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html
- dimitrios1 4y ago
  I would not use Spring, or Java for that matter, for lambdas, speaking from experience. "Lambda containers" is a bit of a misnomer, as you can have multiple instances of a function run on the same container, it's just that initial startup time once the container shuts down that is slow (which can be somewhat avoided by a "warming" function set to trigger on a cron). I would definitely go with containers if your intention is to use Spring. ECS containers can autoscale just the same as lambdas. There's some work being done to package Java code to run more efficiently in serverless computing environments, but IIRC, it's not there yet.
  - afandian 4y ago
    Thanks! I wasn't planning it, but can't hurt to ask. When I looked the Lambda API looked uncomplicated to implement (I saw an example somewhere) and it felt like you could just write a few controllers and gain the ability to run a subset of functionality in Lambda, especially if your app could be kept warm. (to your cron comment, I thought that the reserved capacity would mean the container would be forcibly kept warm?)

daenz 4y ago

They have a feature called "provisioned concurrency" where basically one "instance" of your lambda (or however many you want to configure) stays running warm, so that it can handle requests quickly. I know it defeats the conceptual purpose of serverless, but it's a nice workaround while cloud platforms work on mitigating the cold start problem.

onphonenow 4y ago
I've had some pretty good luck sliming things down as well. That's a win win usually for even non-lambda cases (trying things like docker-slim / switching stuff to go that needs a quick response). That said, the $2-5/month is fine as well for some cases.
catlifeonmars 4y ago
It’s nice to have that dial. Running lambda with provisioned concurrency is still a very managed experience: much different than running a container cluster.
andrew_ 4y ago
That'll also cost you $$$$ and takes any provisioned lambda out of the free tier. Also note that only your specified number of instances will stay warm, meaning if your lambda needs to scale up, you risk slow cold starts on additional instances outside of the number you provisioned. You could specify the number of reserved concurrent images (limiting the number that run) but that also costs money and will eat int your quota. Using containers for lambda is generally a bad idea for anything TypeScript/JavaScript that handles a realtime request - you just can't beat the speed of a single JavaScript file (compiled, in the case of TS). AWS CDK now ships with the NodeJSFunction as well, which makes generating those a breeze with ESBuild.
- daenz 4y ago
  It cost me like $3 a month to get benefit from it. For what it's worth I don't use it for synchronous web requests.

sam0x17 4y ago

If cold starts are at all an issue for whatever use-case, you can just do a warming job like we do (in our case it's built into Ruby on Jets). We find invoking every 30 seconds is enough to never have a cold start. It's still quite cheap as well. The lambda portion of our bill (with tons of platform usage) is still incredibly low / low double digits. Just doing a warming job with no other usage falls well within free tier usage, I can confirm.