3 ms·
> Then, on container startup, your model-serving-app collects the latest model from an external artifact repository. This step usually belong in an init contain
by ellisv 5y ago
> Then, on container startup, your model-serving-app collects the latest model from an external artifact repository. This step usually belong in an init container if you use Kubernetes.
I agree but there are some reasons why folks choose to bake the model into the container image rather than fetching. E.g. time waiting for the model to transfer, artifact repository may be down, etc.
- Fiahil 5y agoIt's mostly because it's easier. No additional token necessary, No config, No Service discovery. For small models (less than 100MB), it's generally not an issue to embed them directly in the image. However, if you're getting GB-size images with less than half of it related to the serving code and its library, then it's time to upgrade.