4 ms·
From a big-O notation perspective, if you're serving N queries, working the kinks out of your routing system costs perhaps O(logN) in developer time, whereas LL
by 0xDEAFBEAD 3mo ago
From a big-O notation perspective, if you're serving N queries, working the kinks out of your routing system costs perhaps O(logN) in developer time, whereas LLM provider savings grow as O(N).
In other words, the more queries you're serving, the more worthwhile it looks to figure out a viable method of model routing.