3 ms·

Lambda is an exciting platform, but it does require some bending to get it to work for certain use cases. This reminds me of some problems we solved last year

Lambda is an exciting platform, but it does require some bending to get it to work for certain use cases. This reminds me of some problems we solved last year [0], which should not have been problems for a fairly straightforward application. [0]: https://iplayer.engineering/evaluating-tensorflow-models-in-aws-lambda-c0e06cf23d87 https://iplayer.engineering/evaluating-tensorflow-models-in-...

ComputerGuru 9y ago
I read that article before and just reread it now - thanks for writing it up, but I still have the same question I did the first time around. You posit: > Because we run 200 invocations or so in parallel we’ll only need to download the model once and save it there From my own reading of the Lambda docs, it seems that a simultaneous request for the same Lambda may or may not spin up a new container, ie while serial requests within the 15 minute timeout will likely reuse the same instance with the same frozen/cached tmp data, parallel requests do not have that assumption. Was this your finding in practice? If so, were your CloudWatch “keep warm” events a series of just 1 Lambda invocation ~10/15min apart of 200 simultaneous requests serially spaced to keep the instances spun up?
- mszczepanczyk 9y ago
  > From my own reading of the Lambda docs, it seems that a simultaneous request for the same Lambda may or may not spin up a new container, ie while serial requests within the 15 minute timeout will likely reuse the same instance with the same frozen/cached tmp data, parallel requests do not have that assumption. This is true. I learned this a hard way some time ago while trying to understand why our lambda function crashes from time to time. It turned out we didn't clean the /tmp folder and we used the space quite extensively. There's a good article on this: https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/ https://aws.amazon.com/blogs/compute/container-reuse-in-lamb...
- niklasrde 9y ago
  The CloudWatch "keep warm" events are just one invocation, yes. I do not remember having had issues with it, but honestly, I don't think I actually have stats on that anymore. I've just checked in S3, but it doesn't look like we have request or data transfer metrics enabled on the model bucket. I may enable those next week to monitor the effectiveness of our strategy better.
  - ComputerGuru 9y ago
    I imagine if the code were properly written to deal with race conditions you wouldn’t notice any issues either way besides an increased latency for some requests. Good luck!

ComputerGuru 9y ago

I read that article before and just reread it now - thanks for writing it up, but I still have the same question I did the first time around. You posit: > Because we run 200 invocations or so in parallel we’ll only need to download the model once and save it there From my own reading of the Lambda docs, it seems that a simultaneous request for the same Lambda may or may not spin up a new container, ie while serial requests within the 15 minute timeout will likely reuse the same instance with the same frozen/cached tmp data, parallel requests do not have that assumption. Was this your finding in practice? If so, were your CloudWatch “keep warm” events a series of just 1 Lambda invocation ~10/15min apart of 200 simultaneous requests serially spaced to keep the instances spun up?

mszczepanczyk 9y ago
> From my own reading of the Lambda docs, it seems that a simultaneous request for the same Lambda may or may not spin up a new container, ie while serial requests within the 15 minute timeout will likely reuse the same instance with the same frozen/cached tmp data, parallel requests do not have that assumption. This is true. I learned this a hard way some time ago while trying to understand why our lambda function crashes from time to time. It turned out we didn't clean the /tmp folder and we used the space quite extensively. There's a good article on this: https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/ https://aws.amazon.com/blogs/compute/container-reuse-in-lamb...
niklasrde 9y ago
The CloudWatch "keep warm" events are just one invocation, yes. I do not remember having had issues with it, but honestly, I don't think I actually have stats on that anymore. I've just checked in S3, but it doesn't look like we have request or data transfer metrics enabled on the model bucket. I may enable those next week to monitor the effectiveness of our strategy better.
- ComputerGuru 9y ago
  I imagine if the code were properly written to deal with race conditions you wouldn’t notice any issues either way besides an increased latency for some requests. Good luck!