3 ms·
Be careful about unofficial providers, a lot of them misconfigure models or stealth quantize them. For a while the difference between Kimi on the official API a
by CuriouslyC 4mo ago
Be careful about unofficial providers, a lot of them misconfigure models or stealth quantize them. For a while the difference between Kimi on the official API and most third party providers was 20-40%.
- unrvl22 4mo agothe 2 I mentioned both have a fairly large following, who run benchmarks and absolutely will spot issues.
- cedws 4mo agoOpenRouter should be penalising or banning for this.
- kilroy123 4mo agoThis is my biggest complaint about OpenRouter and I'm a fan. Might be pretty tough at scale?
- alecco 4mo agoWould that align with their VC-backed incentives?
- mrngld 4mo agoIf your users can't trust your product then I'd say that'd be a pretty strong incentive?
- orbital-decay 4mo agoThey have an "exacto" category with providers they supposedly verified
- ComputerGuru 4mo agoThat’s only for tool use.
- rsanek 4mo agoThat's just one part of it. According to https://openrouter.ai/docs/guides/routing/model-variants/exacto https://openrouter.ai/docs/guides/routing/model-variants/exa... We use three classes of signals: * Tool-calling success and reliability from real traffic * Provider performance metrics such as throughput and latency * Benchmark and evaluation data as it becomes available
- thehamkercat 4mo agoKimi K2 had a vendor verifier: https://github.com/MoonshotAI/K2-Vendor-Verifier https://github.com/MoonshotAI/K2-Vendor-Verifier (there's a table which shows comparison between vendors) Also, it seems there's a general one as well (for all kimi models?): https://github.com/MoonshotAI/Kimi-Vendor-Verifier https://github.com/MoonshotAI/Kimi-Vendor-Verifier
- pranavj 4mo ago[flagged]