4 ms·

Waiting on GPU lambdas to be launched for a lot of ML projects!

• bvm 5y ago
  so...my use case for GPU lambdas is for inference. Say 500 inferences/day ~1 sec/inference, has to be real time as the output is user facing. Can they really spin up quick enough to actually respond in real time? I don't really want to have to keep them warm as I'll end up paying through the nose for it.
  • DarthNebo 5y ago
    For some of my use cases the real-time response isn't essential, just needs one of containers for training. I figured I can just go about ECS, but thought if these were wrapped as lambdas it would just be a matter of toggling the base-image/lambda type.