4 ms·
Interesting approach. I built something similar (Komilion) — a model router that classifies each request and routes to the cheapest model that handles it well.
by robinbanner 9mo ago
Interesting approach. I built something similar (Komilion) — a model router that classifies each request and routes to the cheapest model that handles it well.
A few things I learned building this:
1. Price-only routing is not enough. You need quality signals too. I use LMArena and Artificial Analysis benchmarks to set quality floors per tier.
2. A regex fast path for simple queries (greetings, yes/no, translations) handles ~60% of requests in <5ms with zero API calls to a classifier. This matters because routing latency eats into your savings if every request hits an LLM classifier first.
3. Provider reliability varies wildly. Auto-failover saves you from the "DeepSeek is down" problem — your app keeps working, just on the next cheapest option.
One real example: a support bot went from ~$250/mo on Opus 4.6 to ~$40/mo with routing. Most conversations were FAQ-level.
https://www.komilion.com https://www.komilion.com if anyone wants to try it (free tier, no card).