4 ms·
I have suspected for a long time that hosted models load shed by diverting some requests to lesser models or running more quantized versions under high load.
by api 1y ago
I have suspected for a long time that hosted models load shed by diverting some requests to lesser models or running more quantized versions under high load.
- deleted 1y ago[deleted]
- parineum 1y agoI think OpenRouter saves tokens by summarizing queries through another model, IIRC.