3 ms·
> a real-time router that quickly decides which model to use based on conversation type, complexity, tool needs, and explicit intent This is sort of interestin
by justusthane 1y ago
> a real-time router that quickly decides which model to use based on conversation type, complexity, tool needs, and explicit intent
This is sort of interesting to me. It strikes me that so far we've had more or less direct access to the underlying model (apart from the system prompt and guardrails), but I wonder if going forward there's going to be more and more infrastructure between us and the model.
- hirako2000 1y agoConsider it a low level routing. Keeping in mind it allows the other non active parts to not be in memory. Mistral afaik came up with this concept, quite a while back.
- ItsHarper 1y agoIt's actually just a high-level routing between the reasoning and non-reasoning models that only applies to ChatGPT.
- ItsHarper 1y agoThat only applies to ChatGPT. The API has direct access to specific models.
- willy_k 1y agoThe router seems to only apply to the ChatGPT version, not the API, so it’s not really anything new. Gemini already has functionally dynamic reasoning effort.