3 ms·
Founder of Grafbase here. Here are a few key differentiators vs LiteLLM today: - Nexus does MCP server aggregation and LLM routing - LiteLLM only does LLM rou
by fbjork 1y ago
Founder of Grafbase here.
Here are a few key differentiators vs LiteLLM today:
- Nexus does MCP server aggregation and LLM routing - LiteLLM only does LLM routing
- The Nexus router is a standalone binary that can run with minimal TOML configuration and optionally Redis - LiteLLM is a whole package with dashboard, database etc.
- Nexus is written in Rust - LiteLLM is written in Python
That said, LiteLLM is an impressive project, but we're just getting started with Nexus so stay tuned for a steady barrage of feature launches the coming months:)
- SparkyMcUnicorn 1y agoWhat's the difference between "MCP Server Aggregation" and the litellm_proxy endpoint described here? https://docs.litellm.ai/docs/mcp https://docs.litellm.ai/docs/mcp
- tomhoule 1y agoThe main difference is that while you can get Nexus to list all tools, by default the LLM accesses tools by semantic search — Nexus returns only the relevant tools for the what the LLM is trying to accomplish. Also, Nexus speaks MCP to the LLM, it doesn't translate like litellm_proxy seems to do (I wasn't familiar with it previously).