11 ms·

I don't think based on my tests that lambda has enough CPU to deal with that. The current size I am using is pushing what they are able to process. Thats just b

by boyter 5y ago

I don't think based on my tests that lambda has enough CPU to deal with that. The current size I am using is pushing what they are able to process. Thats just based on my tests however. I do know that the same index on a physical machine can process 200 million items in ~100ms just to give an idea. On lambda it was more like 100,000. Keep in mind I stopped looking at that point. If someone is able to prove otherwise id be happy to switch over to this, but then again I don't have a real need currently.

Dunedan 5y ago
What is the maximum memory size you tested the AWS Lambda functions with? From your article it seems you did try with 1024MB. Did you try with more memory as well? In case you don't know, you need at least 1769MB of memory to get a full vCPU [1]. When running locally on your physical machine, is your code only utilizing a single CPU core or does it make use of multiple cores? If it's the latter one, increasing the memory for the AWS Lambda function even further should improve performance as well. Also if you hit a performance plateau with a memory size of less than 1769MB for your AWS Lambda functions, you could be bottlenecked by something else. I can imagine details like memory bandwidth playing a role there as well. [1]: https://docs.aws.amazon.com/lambda/latest/dg/configuration-function-common.html#configuration-memory-console https://docs.aws.amazon.com/lambda/latest/dg/configuration-f...
- boyter 5y ago
  I tried up to the maximum a few times. The code is single threaded. It is indeed also memory bandwidth bottlenecked. Which I think might be the main issue with it in lambda. My guess based on my tests is that memory access and cpu are the limiting factors in lambda for this but that because it encourages scale out less of an issue.

11 ms·

I don't think based on my tests that lambda has enough CPU to deal with that. The current size I am using is pushing what they are able to process. Thats just b

by boyter 5y ago

Dunedan 5y ago
What is the maximum memory size you tested the AWS Lambda functions with? From your article it seems you did try with 1024MB. Did you try with more memory as well? In case you don't know, you need at least 1769MB of memory to get a full vCPU [1]. When running locally on your physical machine, is your code only utilizing a single CPU core or does it make use of multiple cores? If it's the latter one, increasing the memory for the AWS Lambda function even further should improve performance as well. Also if you hit a performance plateau with a memory size of less than 1769MB for your AWS Lambda functions, you could be bottlenecked by something else. I can imagine details like memory bandwidth playing a role there as well. [1]: https://docs.aws.amazon.com/lambda/latest/dg/configuration-function-common.html#configuration-memory-console https://docs.aws.amazon.com/lambda/latest/dg/configuration-f...
- boyter 5y ago
  I tried up to the maximum a few times. The code is single threaded. It is indeed also memory bandwidth bottlenecked. Which I think might be the main issue with it in lambda. My guess based on my tests is that memory access and cpu are the limiting factors in lambda for this but that because it encourages scale out less of an issue.