4 ms·
I think what the parent is saying is that the model itself has the best context for whether a portion of a request should be routed. The specifics of that routi
by posix_compliant 2mo ago
I think what the parent is saying is that the model itself has the best context for whether a portion of a request should be routed. The specifics of that routing (e.g., should you route to KimiK2) are something that can be trained, finetuned, or even included in a model's startup context.
- johnfn 2mo agoThis doesn't seem quite right. For one, I don't need all the intelligence of an expensive model like Opus 5 to do the relatively simple task of choosing a correct model for a task. Additionally, since this isn't something Anthropic would ever put effort into doing well, you could tune a model to do better and faster than Opus 5 does out of the box.
- kolinko 2mo agoAre you speaking from experience? My experience is the opposite - for many cases it’s not very obvious how good a model needs to be to solve it. Worse models tend to just follow their first instincts without proper reasoning And also btw you don’t need a routing company to decide, you can do it on your harness. And yeah my Fable has zero issues delegating to Terra instead of Opus.
- dannyw 2mo agoAre you thinking of single shot? For agentic use cases, your ability to manage cache and context becomes much more important. Price per individual request becomes less about tokens and more about cache.
- ai-x 2mo agoSmarter models will always route better because it is an intelligent/judgement activity