4 ms·

Is it a bad idea to use AWS Lambda to serve ML models? If cold-start latency is not much of an issue.

• freeqaz 6y ago
  It depends. Do you need a GPU? If so, your options are more limited. If not, then it's definitely doable!
• komuher 6y ago
  No GPU so ye its very bad :P
  • yandie 6y ago
    Depends on whether you need GPU for model serving. Most companies I talk to use CPU for serving and GPU for training, so it's not a bad idea
    • martindbp 6y ago
      OP here, I was thinking more CPU for inference. Training happens elsewhere on GPUs.
      • yandie 6y ago
        Just mean that you'll need to deal with latency, either through provisioned capacity lambda or build it into your service SLA :)
• dcu 6y ago
  We use it for inference and it works fine. Now with EFS support it's even better. We have used it successfully with both Tensorflow and Gorgonia