4 ms·

Depends on whether you need GPU for model serving. Most companies I talk to use CPU for serving and GPU for training, so it's not a bad idea

• martindbp 6y ago
  OP here, I was thinking more CPU for inference. Training happens elsewhere on GPUs.
  • yandie 6y ago
    Just mean that you'll need to deal with latency, either through provisioned capacity lambda or build it into your service SLA :)