3 ms·
The one thing I haven't seen discussed in the serverless world is state as optimization. For instance, if I have a machine learning application that has to acc
by joefkelley 10y ago
The one thing I haven't seen discussed in the serverless world is state as optimization.
For instance, if I have a machine learning application that has to access a very large trained model, I would like to load that model into memory at application startup. Loading from disk at every "function call" would be too slow. So would making RPC calls to some external "model cache" service.
Does AWS Lambda / similar have some sort of overridable "init"?
- jedberg 10y agoThat would be a case where you wouldn't want to use a purely serverless model. You'd want a persistent server that loads the data and then exposes it as an API for your other serverless functions. That being said, it's important to remember that Lambda doesn't reload your function for every function call. It loads it on the first call and then keeps it active for a few minutes as long as there are still requests coming in, so you'd only have to deal with that initial load once in a while (or almost never if your function is active enough). Unless you had a way of extracting your model into a very fast database.