5 ms·

Optimizing Lambda Cost with Multi-Threading

78 points by kiyanwang 6y ago

vcz_fr 6y ago
On the subject of AWS Lambda cost and time optimization: also check out AWS Lambda Power Tuning [^1] and its ecosystem of projects. It also supports 1ms billing and 10GB of RAM. [^1]: https://github.com/alexcasalboni/aws-lambda-power-tuning https://github.com/alexcasalboni/aws-lambda-power-tuning
WatchDog 6y ago
When is Lambda going to catch up to cloud run, and support handling multiple requests concurrent on a single instance.
- yagizdegirmenci 6y ago
  Never.
- petercooper 6y ago
  That's not the model. Lambda is function oriented and handles the concurrency on your behalf. The new container support is really just about providing another delivery mechanism for functions - the underlying model remains the same. If you want something more Cloud Run-esque on AWS, you could use Fargate to rig it up.
  - AzzieElbab 6y ago
    At this point, other than degree of freedom of implementation, what exactly is the real difference between fargate and lambda?
nithril 6y ago
This article sounds like a truism.
koblas 6y ago
My best take away is that if you're running the usual collection of interpreted languages (JavaScript/Python/etc.) you'll probably never get the benefit of having multi-threaded lambda functions for reducing runtime. Given that we now have milisecond billing, this can really result in real savings at scale.
danuker 6y ago
Have you ever considered a different provider than Amazon? Many are significantly cheaper. https://alternativeto.net/software/amazon-web-services-lambda/ https://alternativeto.net/software/amazon-web-services-lambd...
eloff 6y ago
Or you know, don't use lambda. If you need that kind of computer power spinning up a large instance, running the job, and then shutting it down is completely viable. They have per-second billing now on AWS, and you can save even more by using spot instances. Lambda is not cost effective in the CPU/RAM that you get compared to regular instances. It only comes out ahead for very intermittent workloads.
- Autowired 6y ago
  Your points are valid if your priority is to reduce cost. Totally agree. That said, Lambda has an API and SDK, and concurrency is managed. Warm up is a nightmare if you don't develop specifically for Lambda, but scalability is easy. There are trade-offs, it sucks for many use cases, but I don't think you should just shrug it off.
  - eloff 6y ago
    If you have to use beefy lambda instances with multi-threaded workloads? I don't think lambda is the right tool for the job there. I'm not saying it has no value - but that's the opposite of playing to its strengths.
    - Autowired 6y ago
      It all depends, I think. I once used multi-threading to speed up the initialization of a Lambda which was just a "port" of a beefy rest service. It was ported to cut costs and for fault isolation. I managed to gain 2-3 seconds of initialization time, but it was something. A Lambda deployment has the advantage that no one needs to manage it, and faults are isolated between clients. You could argue that if you need these properties your engineering organization has deeper flaws, and you'd be right, but in many companies engineers need to make do with what they get.
      - eropple 6y ago
        > A Lambda deployment has the advantage that no one needs to manage it I assure you that somebody does need to manage it, and if nobody is you are going to eventually hit the wall at speed. I've been called into SEV-1 issues where a Lambda-based web app spontaneously disappeared. (Seasoned AWS heads can probably make a guess as to why.)
SV_BubbleTime 6y ago
I think it would have been helpful to have a chart that showed the final costs for that sample workload.
tomnipotent 6y ago
This is just a benchmark of ffmpeg using lambda and different cpu/thread settings, so it's really "Optimizing ffmpeg on lambda".
coward8675309 6y ago
The exasperating thing about AWS is that it inevitably turns engineering teams into equities and derivative traders. Yes, all programming is about balancing storage, CPU, memory, network I/O against each other, but AWS introduces the complexity of needing to reason about these things over time horizons that are long by a startup's standard. If all of your business's metrics are sharply up and to the right, you probably don't need to worry so much, but if your goal is operational cost efficiency for a relatively stable workload, AWS is a challenge. That may be a blessing in disguise, as self hosting becomes a reasonable thing to consider if your needs are nontrivial and relatively inelastic.
mikesabbagh 6y ago
Are there many looking for a better serverless solution? How about Stateful Serverless?

78 points by kiyanwang 6y ago

On the subject of AWS Lambda cost and time optimization: also check out AWS Lambda Power Tuning [^1] and its ecosystem of projects. It also supports 1ms billing and 10GB of RAM. [^1]: https://github.com/alexcasalboni/aws-lambda-power-tuning https://github.com/alexcasalboni/aws-lambda-power-tuning

WatchDog 6y ago

When is Lambda going to catch up to cloud run, and support handling multiple requests concurrent on a single instance.

yagizdegirmenci 6y ago
Never.
petercooper 6y ago
That's not the model. Lambda is function oriented and handles the concurrency on your behalf. The new container support is really just about providing another delivery mechanism for functions - the underlying model remains the same. If you want something more Cloud Run-esque on AWS, you could use Fargate to rig it up.
- AzzieElbab 6y ago
  At this point, other than degree of freedom of implementation, what exactly is the real difference between fargate and lambda?

nithril 6y ago

This article sounds like a truism.

koblas 6y ago

My best take away is that if you're running the usual collection of interpreted languages (JavaScript/Python/etc.) you'll probably never get the benefit of having multi-threaded lambda functions for reducing runtime. Given that we now have milisecond billing, this can really result in real savings at scale.

danuker 6y ago

Have you ever considered a different provider than Amazon? Many are significantly cheaper. https://alternativeto.net/software/amazon-web-services-lambda/ https://alternativeto.net/software/amazon-web-services-lambd...

eloff 6y ago

Or you know, don't use lambda. If you need that kind of computer power spinning up a large instance, running the job, and then shutting it down is completely viable. They have per-second billing now on AWS, and you can save even more by using spot instances. Lambda is not cost effective in the CPU/RAM that you get compared to regular instances. It only comes out ahead for very intermittent workloads.

Autowired 6y ago
Your points are valid if your priority is to reduce cost. Totally agree. That said, Lambda has an API and SDK, and concurrency is managed. Warm up is a nightmare if you don't develop specifically for Lambda, but scalability is easy. There are trade-offs, it sucks for many use cases, but I don't think you should just shrug it off.
- eloff 6y ago
  If you have to use beefy lambda instances with multi-threaded workloads? I don't think lambda is the right tool for the job there. I'm not saying it has no value - but that's the opposite of playing to its strengths.
  - Autowired 6y ago
    It all depends, I think. I once used multi-threading to speed up the initialization of a Lambda which was just a "port" of a beefy rest service. It was ported to cut costs and for fault isolation. I managed to gain 2-3 seconds of initialization time, but it was something. A Lambda deployment has the advantage that no one needs to manage it, and faults are isolated between clients. You could argue that if you need these properties your engineering organization has deeper flaws, and you'd be right, but in many companies engineers need to make do with what they get.
    - eropple 6y ago
      > A Lambda deployment has the advantage that no one needs to manage it I assure you that somebody does need to manage it, and if nobody is you are going to eventually hit the wall at speed. I've been called into SEV-1 issues where a Lambda-based web app spontaneously disappeared. (Seasoned AWS heads can probably make a guess as to why.)

SV_BubbleTime 6y ago

I think it would have been helpful to have a chart that showed the final costs for that sample workload.

tomnipotent 6y ago

This is just a benchmark of ffmpeg using lambda and different cpu/thread settings, so it's really "Optimizing ffmpeg on lambda".

coward8675309 6y ago

The exasperating thing about AWS is that it inevitably turns engineering teams into equities and derivative traders. Yes, all programming is about balancing storage, CPU, memory, network I/O against each other, but AWS introduces the complexity of needing to reason about these things over time horizons that are long by a startup's standard. If all of your business's metrics are sharply up and to the right, you probably don't need to worry so much, but if your goal is operational cost efficiency for a relatively stable workload, AWS is a challenge. That may be a blessing in disguise, as self hosting becomes a reasonable thing to consider if your needs are nontrivial and relatively inelastic.

mikesabbagh 6y ago

Are there many looking for a better serverless solution? How about Stateful Serverless?