4 ms·
The keep warm time isn't static AFAIK but some frameworks like Zappa have chosen 5 minutes (I work for AWS). If you raise an issue with support there may be oth
by ranman 10y ago
The keep warm time isn't static AFAIK but some frameworks like Zappa have chosen 5 minutes (I work for AWS). If you raise an issue with support there may be other ways around this. There are also other reasons for not relying on functions being warm (spike in concurrent invocations, AZ outages, latency, etc.). If you open a support ticket and email me the # I'll see what I can find out: randhunt at amazon dot com
I'm curious what sort of things you did to shrink your model?
Have you considered pulling the model data from S3 outside of the main lambda handler and seeing if that negatively impacts performance -- with a cloud watch event running every 5 minutes or so to keep the function warm?
- niklasrde 10y agoHey, thanks for your reply. Hadn't thought about a CloudWatch even to keep the function warm, I might suggest that to the team. We haven't shrunk the model, we've deleted superfluous files from the TensorFlow python library and dependencies (we don't need tensorboard, for example). It would be nice if you could package TensorFlow up into a minimal component just for assessment and not have any of the 'learning' stuff or other added-on libraries, but we couldn't find a simple way of doing that - we're not pro C++ engineers and even our python is not the greatest. We're managing for now, but if our model grows any bigger we will run into issues, but there have been some good suggestions in here. We had considered the S3 store, but we ruled that out quite quickly for cost & performance at the number of invocations a months we're looking at - but that was before we knew more about the 'keeping warm', so that may be revisited, too.