2 ms·
Why do you need 10? One or two fallbacks is sufficent. Maybe my workloads are vastly different than yours but I’m not a huge fan of having to switch between mod
by VladVladikoff 1mo ago
Why do you need 10? One or two fallbacks is sufficent. Maybe my workloads are vastly different than yours but I’m not a huge fan of having to switch between models. Even within a single model determinism sucks, debugging why something worked yesterday and doesn’t work today because a model switch is not fun. I can’t imagine the pain of having our services flip randomly between 10 different models.
- pjankiewicz 1mo agoIt is changing so often that to keep the costs and quality at a pareto level you need to experiment with a lot of different providers. And the pain you are describing should be a normal process in AI. I use model pools where the tasks are optimized for multiple AI providers. For example I was experimenting with GPT Luna and it turned out that the model is good but tool shy so I had to improve the instructions. Now this model is my main model for the chat in my app. Next week it can be a different model.
- everforward 1mo agoIt might just not be for you. I use OpenRouter because I do like being able to quickly check whether a new model works better, but mine is a “human in the loop” dev process so it won’t ruin a day of batch processing or anything. I do like being able to try eg GLM without having to set up a new account. It’s also nice that I don’t have to top up per-model accounts. I think I have a couple accounts with $7 in API credits sitting around. Probably not something I would do for actual business processing, where the stability is dramatically more important than tinkering with new models.