5 ms·

The best cold starts are those which aren't noticed by the user. For my blog search (which runs on Lambda), I found a nice way of achieving that [1]: as soon as

The best cold starts are those which aren't noticed by the user. For my blog search (which runs on Lambda), I found a nice way of achieving that [1]: as soon as a user puts the focus to the input field for the search text, this will already submit a "ping" request to Lambda. Then, when they submit the actual query itself, they will hit the already running Lambda most of the times. And, as others said, assigning more RAM to your Lambda than it actually may need itself, will also help with cold start times, as this increases the assigned CPU shares, too. [1] https://www.morling.dev/blog/how-i-built-a-serverless-search-for-my-blog/ https://www.morling.dev/blog/how-i-built-a-serverless-search...

buzzdenver 5y ago
So effectively you're going to pay double?
- musingsole 5y ago
  You increase the usage of the lambda, sure, but not by double unless all this lambda does is respond to the search. There's a point where this is all semantics when running cloud services with regard to how you've deployed your code behind gateways and lambdas, but most applications have obvious triggers that could warm up a lambda at 10% or less overhead. And even then....lambdas are dirt cheap.
  - wpietri 5y ago
    Lambdas can be dirt cheap if you use them well. But that's always what happens. I recently saw a report about contractor who used them for crawling websites. Their client saw a surprise $12k bill for Lambda, when a simple Scrapy crawler on a low-end instance would have cost them < $100/month for the same load. Why? Because if you spin up a ton of Lambda invocations and have them sit around waiting for the network, you pay for each CPU to sit idle, rather than having just one CPU stay busy managing a bunch of async IO.
    - laurencerowe 5y ago
      Couldn't this also have been handled efficiently in Lambda had it been configured to use batching? (Assuming it was implemented as async invocations driven by a queue of some sort.) It does seem unfortunate that Lambda cannot be configured to support concurrent synchronous invocations for serving HTTP requests concurrently.
      - dralley 5y ago
        This is a bit like asking if you can make a coal-rolling pickup truck more environmentally friendly by letting your friends carpool with you. Technically, yes, but it's kinda missing the point.
        laurencerowe 5y ago
        I guess you're paying a 5x premium for Lambda in the constant usage scenario, a similar premium as on-demand vs spot instances. 4GB m6g.medium $0.0385/hour 4GB Lambda works out at $0.1920/hour It's been a while since I last used queue driven autoscaling groups, but https://aws.amazon.com/blogs/compute/scaling-your-applications-faster-with-ec2-auto-scaling-warm-pools/ https://aws.amazon.com/blogs/compute/scaling-your-applicatio... has startup time of about 4 minutes from scratch or 36s from a paused instance in a 'warm pool.' vs less than a second for Lambda in the original article. So the decision ends up coming down to responsiveness vs cost. Presumably Fargate comes somewhere in between.
    - rmbyrro 5y ago
      This is poor architecture, not Lambda's fault. Same result would be achieved by paying tons of $$ for a high-end machine with dozens of cores, just to scrape one URL per core. They could have implemented asyncIO within just a few Lambdas - heck even one Lambda could certainly handle hundreds, if not thousands of ASYNC jobs at once, just as a cheap machine, as you pointed. They treated Lambda functions as threads or async jobs, while they should be looked at as core processors.
      - 5y ago
websap 5y ago
Have you tried provision concurrency[1], this along with ARM based functions can help you with performance and reduce costs. 1 - https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/ https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...
- 015a 5y ago
  Provisioned concurrency is a bit of a non-starter for cold start latency reduction in user-facing applications. Its a tool in the toolbox, but the problem is: Let's say you set the provisioned concurrency at 10, then you have 11 concurrent requests come in. That 11th request will still get a cold start. PC doesn't scale automatically. The ping architecture of warming up functions does scale (better) in this setup. Its not perfect, but nothing on lambda is; its possible that the ping request for User2 could get routed to a live function, which finishes, then starts handling a real request for User1, User2's real request comes in but this function is busy, so Lambda cold starts another one. That being said, these ping functions can be cheap relative to PC. PC is not cheap; its ~$10/mo/gb/function. With the ping architecture; you'd generally just invoke then immediately exit, so there's very little billed execution time. For sparsely executed functions, the ping architecture is better because PC is billed 24/7/365, whereas ping is billed per invoke (that being said: PC is a much cleaner solution). For high volume functions, PC may be cheaper, but it also won't work as well; if your concurrent executions are pretty stable, then its a great choice, but if its spiky high volume (as most setups are!) then it's hard to find a number which balances both cost and efficacy. A setup which also includes Application Autoscaling, to automatically adjust the PC level with time of day or system load or whatever, would approach a better setup. But, of course, be more work. The other frustrating thing about PC is its price relative to a cloudwatch events cron warming setup. Most sources say Lambdas stay warm for ~5 minutes, which means you only need ~15,000 invocations/month per "faux-Provisioned Concurrency" level. Theorycrafting at 100ms/invocation and 512mb: that's ~$0.012/month. PC is ~500 times more expensive than this. This kind of cron-based ping setup is very easy when PC=1; at 1+, it gets harder, but not impossible: You have a cron configured to execute a "proxy" lambda, which then concurrently invokes the real lambda N times depending on what level of PC you more-or-less want. It's also worth noting that PC cannot be maintained on the $LATEST function version. It can only be maintained on an explicit Function Version or Alias (which does not point to $LATEST). So there's just more management overhead there; you can't just have a CI deploy functions and let AWS do everything else, you have to configure the CI to publish new versions, delete the existing PC config, and create a new PC config, maybe also adjust autoscaling settings as well if you configured it. All automatable, but definitely in the domain of "CloudOps Engineer Job Security"; anyone who tells you AWS will make your life easier is probably working for AWS.
yoava 5y ago
Just use http functions in a Velo by Wix project - you will get 50ms cold start out of the box.
- gunnarmorling 5y ago
  Thanks, will take a look. Can I run Java, or any kind of binary, there? That's why I'm not running this with CloudFlare workers, for instance. An alternative I'm actually considering is fly.io.
  - yoava 5y ago
    Velo runs only JavaScript, but unlike cloudflare, velo runs a full node.js. With that in mind, you can a lot of things, from accessing your database or any other asset on aws (or gcp or azure). You can also use other wix apis, like payment api or the wix data database.
    - rmbyrro 5y ago
      If you need to access a DB in AWS, isn't the networking overhead on every single call going to dominate all cold start performance gains in a few occasional requests in Velo?
aahortwwy 5y ago
> assigning more RAM to your Lambda than it actually may need [...] increases the assigned CPU shares "serverless" indeed

buzzdenver 5y ago

So effectively you're going to pay double?

musingsole 5y ago
You increase the usage of the lambda, sure, but not by double unless all this lambda does is respond to the search. There's a point where this is all semantics when running cloud services with regard to how you've deployed your code behind gateways and lambdas, but most applications have obvious triggers that could warm up a lambda at 10% or less overhead. And even then....lambdas are dirt cheap.
- wpietri 5y ago
  Lambdas can be dirt cheap if you use them well. But that's always what happens. I recently saw a report about contractor who used them for crawling websites. Their client saw a surprise $12k bill for Lambda, when a simple Scrapy crawler on a low-end instance would have cost them < $100/month for the same load. Why? Because if you spin up a ton of Lambda invocations and have them sit around waiting for the network, you pay for each CPU to sit idle, rather than having just one CPU stay busy managing a bunch of async IO.
  - laurencerowe 5y ago
    Couldn't this also have been handled efficiently in Lambda had it been configured to use batching? (Assuming it was implemented as async invocations driven by a queue of some sort.) It does seem unfortunate that Lambda cannot be configured to support concurrent synchronous invocations for serving HTTP requests concurrently.
    - dralley 5y ago
      This is a bit like asking if you can make a coal-rolling pickup truck more environmentally friendly by letting your friends carpool with you. Technically, yes, but it's kinda missing the point.
      - laurencerowe 5y ago
        I guess you're paying a 5x premium for Lambda in the constant usage scenario, a similar premium as on-demand vs spot instances. 4GB m6g.medium $0.0385/hour 4GB Lambda works out at $0.1920/hour It's been a while since I last used queue driven autoscaling groups, but https://aws.amazon.com/blogs/compute/scaling-your-applications-faster-with-ec2-auto-scaling-warm-pools/ https://aws.amazon.com/blogs/compute/scaling-your-applicatio... has startup time of about 4 minutes from scratch or 36s from a paused instance in a 'warm pool.' vs less than a second for Lambda in the original article. So the decision ends up coming down to responsiveness vs cost. Presumably Fargate comes somewhere in between.
  - rmbyrro 5y ago
    This is poor architecture, not Lambda's fault. Same result would be achieved by paying tons of $$ for a high-end machine with dozens of cores, just to scrape one URL per core. They could have implemented asyncIO within just a few Lambdas - heck even one Lambda could certainly handle hundreds, if not thousands of ASYNC jobs at once, just as a cheap machine, as you pointed. They treated Lambda functions as threads or async jobs, while they should be looked at as core processors.
    - 5y ago

websap 5y ago

Have you tried provision concurrency[1], this along with ARM based functions can help you with performance and reduce costs. 1 - https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/ https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...

015a 5y ago
Provisioned concurrency is a bit of a non-starter for cold start latency reduction in user-facing applications. Its a tool in the toolbox, but the problem is: Let's say you set the provisioned concurrency at 10, then you have 11 concurrent requests come in. That 11th request will still get a cold start. PC doesn't scale automatically. The ping architecture of warming up functions does scale (better) in this setup. Its not perfect, but nothing on lambda is; its possible that the ping request for User2 could get routed to a live function, which finishes, then starts handling a real request for User1, User2's real request comes in but this function is busy, so Lambda cold starts another one. That being said, these ping functions can be cheap relative to PC. PC is not cheap; its ~$10/mo/gb/function. With the ping architecture; you'd generally just invoke then immediately exit, so there's very little billed execution time. For sparsely executed functions, the ping architecture is better because PC is billed 24/7/365, whereas ping is billed per invoke (that being said: PC is a much cleaner solution). For high volume functions, PC may be cheaper, but it also won't work as well; if your concurrent executions are pretty stable, then its a great choice, but if its spiky high volume (as most setups are!) then it's hard to find a number which balances both cost and efficacy. A setup which also includes Application Autoscaling, to automatically adjust the PC level with time of day or system load or whatever, would approach a better setup. But, of course, be more work. The other frustrating thing about PC is its price relative to a cloudwatch events cron warming setup. Most sources say Lambdas stay warm for ~5 minutes, which means you only need ~15,000 invocations/month per "faux-Provisioned Concurrency" level. Theorycrafting at 100ms/invocation and 512mb: that's ~$0.012/month. PC is ~500 times more expensive than this. This kind of cron-based ping setup is very easy when PC=1; at 1+, it gets harder, but not impossible: You have a cron configured to execute a "proxy" lambda, which then concurrently invokes the real lambda N times depending on what level of PC you more-or-less want. It's also worth noting that PC cannot be maintained on the $LATEST function version. It can only be maintained on an explicit Function Version or Alias (which does not point to $LATEST). So there's just more management overhead there; you can't just have a CI deploy functions and let AWS do everything else, you have to configure the CI to publish new versions, delete the existing PC config, and create a new PC config, maybe also adjust autoscaling settings as well if you configured it. All automatable, but definitely in the domain of "CloudOps Engineer Job Security"; anyone who tells you AWS will make your life easier is probably working for AWS.

yoava 5y ago

Just use http functions in a Velo by Wix project - you will get 50ms cold start out of the box.

gunnarmorling 5y ago
Thanks, will take a look. Can I run Java, or any kind of binary, there? That's why I'm not running this with CloudFlare workers, for instance. An alternative I'm actually considering is fly.io.
- yoava 5y ago
  Velo runs only JavaScript, but unlike cloudflare, velo runs a full node.js. With that in mind, you can a lot of things, from accessing your database or any other asset on aws (or gcp or azure). You can also use other wix apis, like payment api or the wix data database.
  - rmbyrro 5y ago
    If you need to access a DB in AWS, isn't the networking overhead on every single call going to dominate all cold start performance gains in a few occasional requests in Velo?

aahortwwy 5y ago

> assigning more RAM to your Lambda than it actually may need [...] increases the assigned CPU shares "serverless" indeed