I am a k8 newb, so maybe this is a solved problem or not a k8s problem, but I wish there was better tools for hosting ML models in stg and prd. Prd has high tr

I am a k8 newb, so maybe this is a solved problem or not a k8s problem, but I wish there was better tools for hosting ML models in stg and prd. Prd has high traffic during the day; low traffic at night. Pods must be manually right-sized based on the model and expected traffic. Each model gets its own pod. 20 models = 20 pods = 20 cpu and ram configurations per env. Stg has low traffic (50 queries per day). Hosting a container per pod is expensive. Stg hosting per pod costs ~20% of the production cost (if prd cost $10k/mo, staging costs $2k/mo to service a 1% the traffic). I think this can be fixed in the application layer with 2 very different configurations per env, but it would be nice if this was abstracted away from us so we could focus on tuning the models, not configuring proxies and hosting environments.