5 ms·
Show HN: Kalibr – Autonomous Routing for AI Agents
Hey HN, we’re Devon and Alex from Kalibr (https://kalibr.systems https://kalibr.systems).
Kalibr is an autonomous routing system for AI agents. It replaces human debugging with an outcome-driven learning loop. On every agent run, it decides which execution path to use based on what is actually working in production.
An execution path is a full strategy, not just a model: model + tools + parameters.
Most agents hardcode one path. When that path degrades or fails, a human has to notice, debug, change configs, and redeploy. Even then, the fix often doesn’t stick because models and tools keep changing.
I got tired of being the reliability layer for my own agents. Kalibr replaces that.
With Kalibr, you register multiple paths for a task. You define what success means. After each run, your code reports the outcome. Kalibr captures telemetry on every run, learns from outcomes, and routes traffic to the path that’s working best while continuously canarying your alternative paths. When one path degrades or fails, traffic shifts immediately. No alerts, no dashboards and no incident response.
How is this different from other routers or observability tools?
Most routers choose between models using static rules or offline benchmarks. Observability tools show traces and metrics but still require humans to act. Kalibr is outcome-aware and autonomous. It learns directly from production success and changes runtime behavior automatically. It answers not “what happened?” but “what should my agent do next?”
We’re not a proxy. Calls go directly to OpenAI, Anthropic, or Google. We’re not a retry loop. Failed paths are routed away from, not retried blindly. Success rate always dominates; cost and latency only matter when success rates are close.
Python and TypeScript SDKs. Works with LangChain, CrewAI, and the OpenAI Agents SDK. Decision latency is ~50ms. If Kalibr is unavailable, the Router falls back to your first path.
Think of it as if/else logic for agents that rewrites itself based on real production outcomes.
We’ve been running this with design partners and would love feedback. Always curious how others are handling agent reliability in production.
GitHub: https://github.com/kalibr-ai/kalibr-sdk-python https://github.com/kalibr-ai/kalibr-sdk-python
Docs & benchmarks: https://kalibr.systems/docs https://kalibr.systems/docs
- Antonioromero10 8mo agoAwesome tool been using for a month or so now.
- devonkelley 8mo ago[dead]
- neilmagnuson 8mo agoawesome tool for observation, been using it for a while !
- devonkelley 8mo ago[dead]
- adeebvaliulla 8mo agoThis resonates with a pain I see repeatedly in production agent systems: humans acting as the reliability layer. Most teams I work with hardcode a single “golden path” for agents, then rely on dashboards, alerts, and tribal knowledge to notice when behavior degrades. By the time someone debugs model choice, tool params, or prompt drift, the environment has already changed again. The feedback loop is slow and brittle. What’s interesting here is the explicit shift from observability to outcome-driven control. Routing based on actual production success rather than static benchmarks or offline evals aligns with how reliability engineering evolved in other domains. We moved from “what happened?” to “what should the system do next?” years ago. A couple of questions I’m curious about: - How do you define and normalize “success” across heterogeneous tasks without overfitting to short-term signals? - How do you prevent oscillation or path thrashing when outcomes are noisy or sparse? - Is there a notion of confidence or regret baked into the routing decisions over time? Overall, this feels less like a router and more like an autonomous control plane for agents. If it holds up under real-world variance, this is a meaningful step toward agents that are self-healing rather than constantly babysat.
- devonkelley 8mo agoWow, yes. You nailed the framing. Autonomous control plane is the perfect way to describe Kalibr. Defining success: We don't normalize it. Teams define their own outcome signals (latency, cost, user ratings, task completion, etc). You don't need perfect attribution to beat static configs; even noisy signals surface real patterns when aggregated correctly. Oscillation: Thompson Sampling. Instead of greedily chasing the current best path, we maintain uncertainty estimates and explore proportionally. Sparse or noisy outcomes widen confidence intervals, which naturally dampens switching. Wilson scoring handles the low-sample edge cases without the wild swings you'd get from raw percentages. Confidence/regret: Explicit in the routing math. Every path carries uncertainty that decays with evidence. The system minimizes cumulative regret over time rather than optimizing point-in-time decisions. The gap we're closing is exactly what you mentioned. Self-correcting instead of babysat.
- adeebvaliulla 8mo ago
- roan-we 8mo agoI had developed a side project with AI agents to help me summarize the research papers and extract key citations, and I was repeatedly hitting the same annoying pattern. I would finetune everything with GPT4 to perfection, and then in a couple of weeks, it would start hallucinating references or missing citations. I used to waste my Saturday mornings changing prompts and switching models instead of really using the thing. Kalibr pretty much freed me from that loop. I basically arranged GPT-4 and Claude as two different routes, explained that success means accurate citations that I can verify, and now it just works. Last week, GPT-4 oddly started being very slow on longer papers, and by the time I realized it, the traffic was already automatically diverted to Claude. It's like the difference between caretaking an agent and actually having a tool that remains functional without constant supervision. Honestly, I wish I had discovered this a few months ago hehe
- devonkelley 8mo ago[dead]
- 0to1ton 8mo agoCongrats on the launch !!!
- devonkelley 8mo ago[dead]
- curranadvani 8mo agoAmazing work! This will change AI
- devonkelley 8mo ago[dead]
- atajanbaratov 8mo ago[dead]
- ashishforai 8mo agoAmazing tool, much needed. For last two years 80% of yaps were about reliability , reproducibility and observability! Glad this is being addressed here.
- devonkelley 8mo ago[dead]
- deaux 8mo agoComment section needs a look at @ tomhow
- devonkelley 8mo agoThese are real people dude. I know some of them. Some are users or friends who came to comment on our post. They aren't bots and neither am I. Just new to HN.
- devonkelley 8mo ago@tomhow emailing to verify that I am real, the comments on my post are not bots and I can verify whatever you need me to. Annoying to be flagged when I am just new here and trying to be part of the community.
- deaux 8mo agoNever said you're a bot or that they're not real people. They're mostly people you know or your sockpuppets. Most people who do this are real people. https://news.ycombinator.com/item?id=46719034 https://news.ycombinator.com/item?id=46719034 https://news.ycombinator.com/item?id=46742799 https://news.ycombinator.com/item?id=46742799 https://news.ycombinator.com/item?id=46290617 https://news.ycombinator.com/item?id=46290617 https://news.ycombinator.com/item?id=46487533 https://news.ycombinator.com/item?id=46487533 None of these are "bots". https://news.ycombinator.com/threads?id=lighthouse1212 https://news.ycombinator.com/threads?id=lighthouse1212 https://news.ycombinator.com/item?id=46720287 https://news.ycombinator.com/item?id=46720287 https://news.ycombinator.com/threads?id=Phil_BoaM https://news.ycombinator.com/threads?id=Phil_BoaM All "real people", all posting multiple LLM generated comments.
- devonkelley 8mo ago[dead]
- devonkelley 8mo ago[dead]