6 ms·

> To combat this we wrote a "runWarm" function that kept the API alive at all times. This doesn't really work like you'd expect and isn't recommended, as it on

by jpdb 5y ago

> To combat this we wrote a "runWarm" function that kept the API alive at all times. This doesn't really work like you'd expect and isn't recommended, as it only helps a particular use-case. The reason is that AWS Lambda will only keep a single instance of your function alive. That means if two requests come in at the same time, you'd see a cold start on one of those invocations. Instead, you want to look at something like provisioned concurrency.

VWWHFSfQ 5y ago
Provisioned concurrency is insanely expensive. If you have any kind of a thundering herd access pattern then Lambda is a complete non-starter because of the warm-up and scaling characteristics. We eventually just put an nginx/openresty server on a regular medium EC2 instance and got rid of Lambda from our stack completely and now we're paying about 1/300th the cost we were previously and the performance is infinitely better. I'm sure it has some use-cases in some kind of backoffice task queue scenario, but Lambda is nearly unusable in a web context unless you have a very trivial amount of traffic.
- sneak 5y ago
  ...or a big latency budget? Slow start is fine for a sudden burst for a lot of use cases.
- berkes 5y ago
  > I'm sure it has some use-cases in some kind of backoffice task queue scenario, but Lambda is nearly unusable in a web context unless you have a very trivial amount of traffic. This has been the outcome for me on several projects too. Just use loadbalanced EC2 (or EB, for simplification) and pay for a few instances running 24/7. It's actually cheaper than having a busy lambda in all my cases. The only other case (other than occasional backoffice jobs) would be long-tail stuff: an API endpoint that is used in rare situations: for example the "POST /datatakeout" or "DELETE /subscription/1337" or such. Things that might be heavy, require offbeat tools and so on. We've had them for building PDFs and .docx from reports; a feature used by <2% of the users, yet requiring all sorts of tools, from latex to pandoc.
  - tyingq 5y ago
    I would guess this situation is maintained on purpose by AWS as the upsell reason for Fargate.
  - VWWHFSfQ 5y ago
    Yeah the caveats, gotchas, and workarounds you have to do to get something reasonable running on Lambda are just goofy. At some point we just stopped and wondered why we were punishing ourselves with this stuff. We switched to a traditional webserver on regular EC2 instances and haven't looked back.
  - roland35 5y ago
    Have you run into issues with Lambda with complex tasks? I thought there was a 15 minute limit to tasks, plus a maximum storage size when importing large dependencies, etc?
    - berkes 5y ago
      The latex example did not run entirely on Lambda. Lambda would write a job into a queue (just Postgres), trigger a launch of a beefy ec2 instance, after which a worker on that ec2 picked up the job. Another lambda function would be called by the server itself to shut down the worker when all jobs were done. Kludgy and slow. But it worked and did save some money, because the instance running this latex worker was big and chunky yet utilized maybe 10 hours a month. Lambda was mostly acting as a kldugy load-balancer really.
- ctvo 5y ago
  This is another example of AWS over marketing Lambda. Lambda is horrendously expensive when requests pass a certain level per second. You can graph it against ECS / EC2 to see the point it stops becoming economical. Taking all of this into account, Lambda is then useful for a very small niche: - Tasks that don't care about low P99 latency. These tend to be asynchronous processing workflows, as APIs in the customer request path tend to care about low P99 latency. - Tasks that have a low request per second. Again, these tend to be asynchronous processing workflows. You talk to anyone on the AWS serverless team and the conversation eventually focuses on toil. If you can quantify engineering toil for your organization, and give it a number, the point at which Lambda stops being economical shifts right, but it doesn't change the overall shape of the graph.
  - mhaymo 5y ago
    In my experience cold starts don't affect the p99 if you have substantial traffic, because you have enough lambdas consistently running that cold start rate is ~0.1%. P99.9 also matters though!
    - earthboundkid 5y ago
      If you have substantial traffic, the cost savings of Lambda are gone, and you can just use ECS or something.
  - earthboundkid 5y ago
    I use Netlify serverless functions (which is just a wrapper around AWS Lambda) because it basically fits the criteria for me. I have a low but bursty access pattern that fits into the free tier, and there's a static SPA page that can serve up instantly while the XHR triggers to do the cold start fetch. I don't think I would use it for anything consumer facing though. This is just a backend where an extra 300ms isn't going to make a big difference to the admins.
  - nuerow 5y ago
    > This is another example of AWS over marketing Lambda. Lambda is horrendously expensive when requests pass a certain level per second. I feel this is a gross misrepresentation of AWS Lambdas. AWS lambdas are primarily tailored for background processes, event handlers, and infrequent invocations. This is how they are sold, including in AWS' serverless tutorials. Even though they can scale like crazy, and even though you can put together an API with API Gateway or even Application Load Balancer, it's widely known that if your API handles more more traffic than a few requests per second then you're better off putting together your own service. The rationale is that if you don't need to do much with a handler, or you don't expect to handle a lot of traffic on a small number of endpoints, AWS lambdas offer a cheaper solution to develop and operate. In some cases (most happy path cases?), they are actually free to use. Beyond a certain threshold, you're better off getting your own service to run on EC2/Fargate/ECS/whatever, specially given that once you have a service up and running then adding a controller is trivial.
- wmfiv 5y ago
  Insanely expensive is definitely a flexible term. I think numbers help here. Provisions Concurrency $8.64 / GB / month 256 MB per Lambda (Assuming Python, Ruby, NodeJS, or Rust) $2.16 per Lambda per month A lot of organizations can probably make a good business case for keeping 100s or even 1000s of Lambda's warm. You also don't need to keep them warm 24x7, can get an additional 12% discount using savings plans, and if you're a big guy you get your EDP discount.
  - VWWHFSfQ 5y ago
    I'm sure there are plenty of companies that are happy to throw $5,000/mo at a problem that can be solved better for $250/mo. Not mine though.
    - count 5y ago
      Does your $250/mo include all the ops cost of other solutions?
      - VWWHFSfQ 5y ago
        Yes it does
        slownews45 5y ago
        You should look at offering this as a service perhaps. 2,500 250MB lambdas for $250/month with all AWS guarantees (ie, Multi-AZ, permissioning on every call etc etc) would be pretty compelling I think for folks running intermediate lambda workloads (ie, 5-10K lambadas at a time).
        VWWHFSfQ 5y ago
        I'm not trying to offer it as a service. I'm trying to run my workload in a way that can scale from 0 -> 10,000 request/second in an instant and doesn't cost my company $5,000/month to do so. It's pretty easy if you know what you're doing (or care to figure it out).
x86_64Ubuntu 5y ago
Here is a little AWS doc describing what parent is talking about. Personally, I had confused "provisioned concurrency" with "concurrency limit" since I don't work with cloud stuff outside of hobbying. https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/ https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...
raro11 5y ago
Have your runWarm sleep for 500ms and execute 50 of them concurrently. As long as none of the functions are finished and you start a new one you get a new instance, at least that's what I think. You can get 50 hot instances that way no? I'd rather scale per connections. Have a lambda instance do 50 concurrent requests. Something like https://fly.io https://fly.io but cheaper.

by jpdb 5y ago

VWWHFSfQ 5y ago

Provisioned concurrency is insanely expensive. If you have any kind of a thundering herd access pattern then Lambda is a complete non-starter because of the warm-up and scaling characteristics. We eventually just put an nginx/openresty server on a regular medium EC2 instance and got rid of Lambda from our stack completely and now we're paying about 1/300th the cost we were previously and the performance is infinitely better. I'm sure it has some use-cases in some kind of backoffice task queue scenario, but Lambda is nearly unusable in a web context unless you have a very trivial amount of traffic.

sneak 5y ago
...or a big latency budget? Slow start is fine for a sudden burst for a lot of use cases.
berkes 5y ago
> I'm sure it has some use-cases in some kind of backoffice task queue scenario, but Lambda is nearly unusable in a web context unless you have a very trivial amount of traffic. This has been the outcome for me on several projects too. Just use loadbalanced EC2 (or EB, for simplification) and pay for a few instances running 24/7. It's actually cheaper than having a busy lambda in all my cases. The only other case (other than occasional backoffice jobs) would be long-tail stuff: an API endpoint that is used in rare situations: for example the "POST /datatakeout" or "DELETE /subscription/1337" or such. Things that might be heavy, require offbeat tools and so on. We've had them for building PDFs and .docx from reports; a feature used by <2% of the users, yet requiring all sorts of tools, from latex to pandoc.
- tyingq 5y ago
  I would guess this situation is maintained on purpose by AWS as the upsell reason for Fargate.
- VWWHFSfQ 5y ago
  Yeah the caveats, gotchas, and workarounds you have to do to get something reasonable running on Lambda are just goofy. At some point we just stopped and wondered why we were punishing ourselves with this stuff. We switched to a traditional webserver on regular EC2 instances and haven't looked back.
- roland35 5y ago
  Have you run into issues with Lambda with complex tasks? I thought there was a 15 minute limit to tasks, plus a maximum storage size when importing large dependencies, etc?
  - berkes 5y ago
    The latex example did not run entirely on Lambda. Lambda would write a job into a queue (just Postgres), trigger a launch of a beefy ec2 instance, after which a worker on that ec2 picked up the job. Another lambda function would be called by the server itself to shut down the worker when all jobs were done. Kludgy and slow. But it worked and did save some money, because the instance running this latex worker was big and chunky yet utilized maybe 10 hours a month. Lambda was mostly acting as a kldugy load-balancer really.
ctvo 5y ago
This is another example of AWS over marketing Lambda. Lambda is horrendously expensive when requests pass a certain level per second. You can graph it against ECS / EC2 to see the point it stops becoming economical. Taking all of this into account, Lambda is then useful for a very small niche: - Tasks that don't care about low P99 latency. These tend to be asynchronous processing workflows, as APIs in the customer request path tend to care about low P99 latency. - Tasks that have a low request per second. Again, these tend to be asynchronous processing workflows. You talk to anyone on the AWS serverless team and the conversation eventually focuses on toil. If you can quantify engineering toil for your organization, and give it a number, the point at which Lambda stops being economical shifts right, but it doesn't change the overall shape of the graph.
- mhaymo 5y ago
  In my experience cold starts don't affect the p99 if you have substantial traffic, because you have enough lambdas consistently running that cold start rate is ~0.1%. P99.9 also matters though!
  - earthboundkid 5y ago
    If you have substantial traffic, the cost savings of Lambda are gone, and you can just use ECS or something.
- earthboundkid 5y ago
  I use Netlify serverless functions (which is just a wrapper around AWS Lambda) because it basically fits the criteria for me. I have a low but bursty access pattern that fits into the free tier, and there's a static SPA page that can serve up instantly while the XHR triggers to do the cold start fetch. I don't think I would use it for anything consumer facing though. This is just a backend where an extra 300ms isn't going to make a big difference to the admins.
- nuerow 5y ago
  > This is another example of AWS over marketing Lambda. Lambda is horrendously expensive when requests pass a certain level per second. I feel this is a gross misrepresentation of AWS Lambdas. AWS lambdas are primarily tailored for background processes, event handlers, and infrequent invocations. This is how they are sold, including in AWS' serverless tutorials. Even though they can scale like crazy, and even though you can put together an API with API Gateway or even Application Load Balancer, it's widely known that if your API handles more more traffic than a few requests per second then you're better off putting together your own service. The rationale is that if you don't need to do much with a handler, or you don't expect to handle a lot of traffic on a small number of endpoints, AWS lambdas offer a cheaper solution to develop and operate. In some cases (most happy path cases?), they are actually free to use. Beyond a certain threshold, you're better off getting your own service to run on EC2/Fargate/ECS/whatever, specially given that once you have a service up and running then adding a controller is trivial.
wmfiv 5y ago
Insanely expensive is definitely a flexible term. I think numbers help here. Provisions Concurrency $8.64 / GB / month 256 MB per Lambda (Assuming Python, Ruby, NodeJS, or Rust) $2.16 per Lambda per month A lot of organizations can probably make a good business case for keeping 100s or even 1000s of Lambda's warm. You also don't need to keep them warm 24x7, can get an additional 12% discount using savings plans, and if you're a big guy you get your EDP discount.
- VWWHFSfQ 5y ago
  I'm sure there are plenty of companies that are happy to throw $5,000/mo at a problem that can be solved better for $250/mo. Not mine though.
  - count 5y ago
    Does your $250/mo include all the ops cost of other solutions?
    - VWWHFSfQ 5y ago
      Yes it does
      - slownews45 5y ago
        You should look at offering this as a service perhaps. 2,500 250MB lambdas for $250/month with all AWS guarantees (ie, Multi-AZ, permissioning on every call etc etc) would be pretty compelling I think for folks running intermediate lambda workloads (ie, 5-10K lambadas at a time).
        VWWHFSfQ 5y ago
        I'm not trying to offer it as a service. I'm trying to run my workload in a way that can scale from 0 -> 10,000 request/second in an instant and doesn't cost my company $5,000/month to do so. It's pretty easy if you know what you're doing (or care to figure it out).

x86_64Ubuntu 5y ago

Here is a little AWS doc describing what parent is talking about. Personally, I had confused "provisioned concurrency" with "concurrency limit" since I don't work with cloud stuff outside of hobbying. https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/ https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...

raro11 5y ago

Have your runWarm sleep for 500ms and execute 50 of them concurrently. As long as none of the functions are finished and you start a new one you get a new instance, at least that's what I think. You can get 50 hot instances that way no? I'd rather scale per connections. Have a lambda instance do 50 concurrent requests. Something like https://fly.io https://fly.io but cheaper.