3 ms·
That is interesting, sounds like in practice you only end up routing between 2 models
by alansaber 4mo ago
That is interesting, sounds like in practice you only end up routing between 2 models
- adchurch 4mo agoI'd say that a typical main agent loop has 1-3 models (obviously very situationally dependent), but when you have subagents those can get routed independently since they have a fresh context window, so there are a lot more degrees of freedom there.
- echelon 4mo agoOr not routing at all. In practice you just pick one and stick with it until the API stops or you hit performance issues.
- adchurch 4mo agoThe choice on the first turn is super important for this reason! But if a user prompt sends the convo in a very different direction then often it does make sense to reroute at that point.