2 ms·
That makes a lot of sense for anything that's public-facing and free. If the user is having a dialog with the system, they start out with a trivial question, a
by Animats 2y ago
That makes a lot of sense for anything that's public-facing and free.
If the user is having a dialog with the system, they start out with a trivial question, and are routed to a simple LLM, are they stuck there? Or can the conversation be transitioned to something more powerful?
Something like this will be needed to direct traffic as systems specialize more.
- danlenton 2y agoGreat question! Nope they're not stuck with the simple LLM after the first question. Chatbots simply aggregate all prior conversation history in the input prompt, and make a new LLM call. The router treats each LLM call seperately, and so a single conversation can occur across many models and providers.