3 ms·
Yes, you hit it on the head. It's worth it, but you have to do your own evaluation and promotion. Example I forgot to mention: `:nitro` ranking is not the fast
by mmoustafa 23d ago
Yes, you hit it on the head. It's worth it, but you have to do your own evaluation and promotion.
Example I forgot to mention: `:nitro` ranking is not the fastest, I do a round robin sampling with representative payloads to find out the fastest providers and reorder my list on the fly.
- johnsmith1840 23d agoYour doing that sampling/promotion automatically? I considered an automatic promotion path but decided against it I want to actually review the data myself first. What's your model churn rate like? I was worried about customer experience by same day maybe same work getting a totally different model response (also caching is worse)