4 ms·
How does this work under the hood? Is the model loaded every time it receives a request? Is it run in a docker or a lambda? How does it work after "uploading it
by kuu 7y ago
How does this work under the hood? Is the model loaded every time it receives a request? Is it run in a docker or a lambda? How does it work after "uploading it" to amazon?
- ospillinger 7y agoEach model is loaded into a Docker container, along with any Python packages and request handling code. The cluster runs on EKS on your AWS account. Cortex takes the declarative configuration from 'cortex.yaml' and creates it every time you run 'cortex deploy' so the containers don’t change unless you run 'cortex deploy' again with updated configuration. This post goes into more detail about some of our design decisions: https://towardsdatascience.com/inference-at-scale-49bc222b3ad1 https://towardsdatascience.com/inference-at-scale-49bc222b3a...
- kuu 7y agoThank you!