5 ms·
It took me quite a while to come round to OpenRouter. Originally I didn't understand why anyone would put a proxy between them and an LLM, but it actually adds
by simonw 5mo ago
It took me quite a while to come round to OpenRouter. Originally I didn't understand why anyone would put a proxy between them and an LLM, but it actually adds some quite significant value:
1. By far the lowest friction way to support and try out all the models.
2. They offer billing caps! Most model providers still don't do this [EDIT: maybe they do, see reply comment], but if you're going to run anything in public it's very useful to have hard limits so it doesn't cost you $1m overnight because someone started abusing it.
3. Their rankings are one of the more interesting signals for which models are popular, despite their flaws (most OpenAI and Anthropic users don't go via OpenRouter, it's currently not possible to tell the difference between many users switching v.s. one "whale" changing their preferred model)
Given how API costs are becoming meaningful for a lot of companies now, having a provider like OpenRouter to help measure your spend and easily experiment with and switch providers feels like a valuable service.
- SilverElfin 5mo agoThe biggest benefit is that it creates competition among models. If more people use open weight models or models from other providers, it’ll be harder to ban them. Which is what OpenAI and Anthropic will try to accomplish. OpenAI by lobbying the Trump administration for favorable treatment (see Brockman’s MAGA PAC donations), Anthropic by using religious leaders and nonprofits to push “safety” justifications for difficult regulations.
- fontain 5mo agoOut of interest, why OpenRouter over a free option like Cloudflare’s AI gateway or another paid option like Vercel’s — any specific benefit to OpenRouter you’ve found, or just first you used that’s good enough?
- simonw 5mo agoI'll be honest, I hadn't clocked that Cloudflare and Vercel were offering equivalent products. Looks like Vercel even have their own leaderboard: https://vercel.com/ai-gateway/leaderboards/models https://vercel.com/ai-gateway/leaderboards/models Surprising that they have Opus 4.8 and 4.6 listed on the leaderboard but not Opus 4.7.
- c-hendricks 5mo agoHuh, Claude Opus 4.8 is number 4 in number of tokens at 10%, not even in the top 10 in terms of requests, yet is #1 in costs at a whopping 43%!
- deleted 5mo ago[deleted]
- zenoprax 5mo agoI didn't know about these options either. I am using Cline: Cloudflare isn't an option but Vercel is. My spending is pretty low overall now that I'm using local models much more but good to know that there are cheaper alternatives to try or at least suggest to others. Other features I've just noticed: - configurable prompt injection protection using OWASP regex (https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_Injection_Prevention_Cheat_Sheet.html https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_In...) - configurable PIM protection for outbound prompts - input/output logging - "JSON healing" to auto-correct minor hallucinations Lots of other stuff too. The business model seems pretty simple and the value-add features don't look particularly expensive or difficult to copy.
- js4ever 5mo agoSeparation, I don't want to have my domains blocked the day AI bill go Brrr
- wahnfrieden 5mo agoMake two accounts...
- yencabulator 5mo ago> a free option like Cloudflare’s AI gateway Free? They take the same 5% fee as OpenRouter does. https://developers.cloudflare.com/ai-gateway/features/unified-billing/ https://developers.cloudflare.com/ai-gateway/features/unifie...
- Aurornis 5mo agoGood points. The easy experimentation factor is helpful for development, though I would gently encourage everyone to migrate to the 1st party APIs for pricing at scale. OpenRouter is also a good place to find free LLM access with a catch: You should expect that any inputs and outputs are going into someone's training database. Clearly anyone who can pay should be using paid models with privacy protections, but the free models have been great for learning and experimenting. Especially for younger people learning API programming and LLMs who may not have access to a credit card or funds.
- bix6 5mo agoIt’s interesting all the focus on opt-out from training. Sometimes I worry there is an intentional focus on that so people don’t think about the other ways the company might be profiting off our data. Like I pay for Anthropic and they don’t train on that but are they selling my “anonymized” usage data in some other way?
- derefr 5mo agoFrom what I recall, these companies don't offer any option to opt out of your session transcript data being used (and sold!) for "regular" adtech targeting purposes.
- nl 5mo agoAnthropic explicitly state that they don't do this, even if you use the free plan and even if you don't opt-out of letting them use your data for training: "We do not sell users’ data to third parties." https://www.anthropic.com/news/updates-to-our-consumer-terms https://www.anthropic.com/news/updates-to-our-consumer-terms
- derefr 5mo agoThat answers for the "sold" part but not for the "used" part. I.e. nothing about this statement prevents Anthropic from running ads within Claude, as long as they run the ad-placement auctions themselves, and so aren't leaking any of the data they're using to decide which placements are relevant to which users+sessions. (This is the same thing Google does for SERP ad auctions.) But actually, and perhaps more interestingly, nothing about this statement prevents Anthropic from building a Google AdSense competitor either. Other sites (or mobile apps, etc) could plop in an Anthropic ad iframe; and it'd be Anthropic's knowledge of your interactions with Claude that would drive what ads would show up in that iframe. The embedding site doesn't know what ads the users are seeing, so that's still not "selling users' data to third parties", per se.
- maxloh 5mo agoOpenRouter is merely only a proxy. They also host some open-weight models
- simonw 5mo agoI don't think they do. They proxy to a bunch of open-weight model hosts, but I've not seen that they host them themselves. They don't list themselves on https://openrouter.ai/providers https://openrouter.ai/providers
- minimaxir 5mo agoOpenRouter has stealth models that are indicated as "by OpenRouter" but indicate an external provider. https://openrouter.ai/openrouter/owl-alpha https://openrouter.ai/openrouter/owl-alpha
- zorked 5mo agoThe way how you manage the caps in OpenRouter is how every metered API provider should do it: keys have limits, and you can change the limits, and you set the limits to refill periodically, and you can create as many keys as you want.
- a13n 5mo agoBoth OpenAI and Anthropic have billing caps… who doesn’t?
- totaa 5mo agoGoogle Vertex
- srameshc 5mo agohttps://aistudio.google.com/spend https://aistudio.google.com/spend ? Monthly spend cap
- mips_avatar 5mo agoI spent two hours the other day trying to figure out how to manage spend on gcp, i gave up and used openrouter and cloudflare.
- squeaky-clean 5mo ago> Long-running tasks like batch mode completions and agent sessions may incur overages beyond your project spend cap. > Billing data processing times can be delayed in AI Studio, up to around 10 minutes. You may experience overages beyond your project cap if billing data hasn't processed before more charges are accrued. https://ai.google.dev/gemini-api/docs/billing#project-spend-caps https://ai.google.dev/gemini-api/docs/billing#project-spend-... That's a soft cap, not a hard cap
- deleted 5mo ago[deleted]
- movedx01 5mo agoAI studio added it recently, Vertex not.
- deleted 5mo ago[deleted]
- simonw 5mo ago
- alecco 5mo agoAt the moment for DeepSeek V4 it messes up caching and that's a key pricing feature for V4. https://news.ycombinator.com/item?id=48319827 https://news.ycombinator.com/item?id=48319827
- greedydecode 5mo ago[flagged]
- GodelNumbering 5mo agoAnother neat thing is, they publish hourly caching states for ALL model/provider combinations. I did some research on it to come up with a provider tiers list and found a bunch of open-source 3rd party hosts are simply trash tier https://dirac.run/posts/cache-hit-rates-agents https://dirac.run/posts/cache-hit-rates-agents
- gnulinux 5mo agoThank you so much for this! I've been working on exactly this problem this week (which OpenRouter providers have the highest cache rate on average) because cache cost is sometimes half your cost: I'd much rather use a provider with more input caching with a more expensive/better LLM. Your results and lists seem more comprehensive than what I've done so far. Very helpful!
- rkagerer 5mo agoAgents push the full conversation history into context every turn Why? Maybe this is a dumb question, but why wouldn't an agent "keep the conversation going", like I do when interacting with an LLM through a web page? (I understand how it's impractical for long-running tasks where the agent has to wait days for the next input, but assume that's not the majority of use cases)
- BoredPositron 5mo agoThe "web page" does the same you just don't see it.
- sosodev 5mo agoI’m not sure I understand your question. Every interaction you have with a model in a web page does the same thing in the backend. It feeds the whole conversation history, perhaps with a bit of processing, into the model so it can process the next generation. Filling the context window is how these models retain coherence.
- isbvhodnvemrwvn 5mo agoLLMs are stateless, to predict next tokens they need the history. When you write your own agents you will be very selective and might trim context and heavily segment different tasks, but generic ones don't do that (at best they spawn subjects to handle smaller tasks)
- stymaar 5mo agoAnd what is their business model?
- minimaxir 5mo agoCredit is prepaid at a 5% surcharge.
- MangoCoffee 5mo agomiddle man? Model providers/hyperscalers -> OpenRouter -> consumers? coffee farmers -> middle man -> you
- behnamoh 5mo agoCommission on API calls extracted from you when you charge your account.
- arw0n 5mo agoI love their product and use them myself. But where's the value proposition for investors? Unless they get purchased by one of the large cloud providers, they will get pushed out of the market sooner or later. What's the value proposition for the typical AWS startup to go with openrouter, if Amazon offers similar rates with direct integration into all their other offerings? The only reason OpenRouter can exist at the moment is because we are in the wild-west phase of this technology, and lots of people and companies are exploring. In 5 years they will have to have transformed their business fundamentally, or go the way of the dinosaurs.
- rat9988 5mo agoThey never claimed it was technically hard. Brand recognition is their forte. They found out there is a need, developped a product around it.
- brianwawok 5mo agoYah I don’t think they have a long term play without a pivot
- rsalus 5mo agofunctionally they operate as a marketplace for cloud providers. I feel like there is value there, especially as API costs rise and companies explore cost-saving/efficiency. IMO, this is a particularly attractive value prop in the SMB space, where it is common to interoperate between multiple SaaS/software stacks.
- pizzly 5mo agoAWS does not provide nearly as many different models as OpenRouter. Perhaps they have an incentive to not do that, move slower as a big company or more legal risks to consider. If AI model outputs becomes commoditized then having one place where you can switch effortlessly from one to the next based on price might just justify OpenRouter. It could become a commodity marketplace/exchange.
- sowbug 5mo agoIf you believe there will be lots of LLM providers in the future, then OpenRouter could be a DoorDash play. Established restaurants didn't need DoorDash because they were already on everyone's speed dial. But new or small restaurants couldn't afford to advertise or maintain a team of delivery people. DoorDash created a two-sided marketplace that made it a lot easier for new entrants to bootstrap. Today even the established restaurants have to pay them their tithe because hungry people have learned to start with the DoorDash app. A bit of a prisoner's dilemma. If OpenRouter plays its cards right and gets very lucky, a large number of people will configure their hungry LLM clients to start with OpenRouter, and then LLM providers will have to join the marketplace or else miss out on all those customers.
- brianwawok 5mo agoI’m just not sure they have a moat or a long term play? I put $20 in and tried a few models. Then I went right to the model provider to put in $1000 and avoid the middleman tax. Now imagine a big corp spending millions on AI. That’s a lot of middleman tax.
- brianjking 5mo agoI tend to agree, but there's also a lot of tax to build and maintain the different provider abstractions that OpenRouter eliminates. Everything has a cost of some sort. It's just who you're going to pay and what the currency is.
- BoredPositron 5mo agoThere are enough services that don't want the model provider to know who they are.
- polski-g 5mo agoAnd what do you do when Fireworks is down? If you stuck with Openrouter, when Fireworks is down it would auto route you to Friendli. What if Fireworks stops offering your preferred model?
- brianwawok 5mo agoHonestly I am 98% on Claude, and when claude is down I suffer through GPT.
- TurdF3rguson 5mo agoThe top model / prices are changing all the time though. Lately I've been auditioning 4-5 models before a big ingest and I wouldn't be able to do that easily without OR.
- kristianp 5mo agoThe value of openrouter isn't as a middleman for users of claude, gemini or chatgpt, it's for those looking to find a model that fills the use case at a lower price than the top 3.
- MillionOClock 5mo agoBilling caps are underrated! I don't understand why they aren't present everywhere. As an indie dev there are some services I'm really hesitant on trying by fear of getting an enormous bill for a mistake, this is even more true with vibe coding IMO.
- JumpCrisscross 5mo ago> By far the lowest friction way to support and try out all the models Check out Kagi Ultimate.
- MicrosoftShill 5mo agoWould you recommend Kagi Ultimate over OpenRouter? I'm already a customer of Kagi and would rather give them my money, but only if I'm not really compromising.
- JumpCrisscross 5mo ago> Would you recommend Kagi Ultimate over OpenRouter? For personal use, yes. The all-in pricing model encourages experimentation. And the privacy pitch seems tighter.
- MicrosoftShill 5mo agoThe privacy part is definitely important. Appreciate you! Maybe someday the VM I run agents in will have a dedicated GPU so that I can stop using APIs altogether. One can dream...
- HDBaseT 5mo agoHow do you use Kagi Ultimate for programming though? The difference between GLM 5.1, ChatGPT 5.4 and Opus don't really matter to me when I'm asking/talking, but programming is a different beast that needs a harness.
- jampekka 5mo agoThe main friction reduction, for me at least, is the consolidated billing that avoids extra bureaucracy in corporate environments. The API-translation/abstraction tends to cause more problems than it solves. I’d prefer something that consolidates billing, but still lets me use providers' APIs directly (or via some "raw HTTP" proxy). There are plenty of unified API gateways, but I haven’t seen one that is just billing/auth in front of the native provider APIs.
- voidstitch 5mo ago[flagged]
- deleted 5mo ago[deleted]
- sarjann 5mo agoThere is also the ability to fallback is one of the clouds degrades in performance.
- what 5mo agoWhat kind of compensation are they giving you?
- simonw 5mo agoNone. It's sometimes possible for people to say something positive about a project without being paid! (If they were paying me they got a bad deal, since I called out the flaws in their leaderboard approach half way through my post.)
- scosman 5mo agoThey also do a good job working over the little differences between APIs. Tool calling sometimes breaks on major providers, and OR will patch it before the provider does. Libraries like LiteLLM do this too, but OR is faster.
- octoberfranklin 5mo agoUnfortunately the model companies will simply reinject the friction by mandating BYOK (Bring Your Own Key -- i.e. the end user must onboard with each model company individually). OpenAI and Anthropic have already done this. Mandated BYOK will sink OpenRouter.
- mrtksn 5mo agoDid you know that if you put some money into your OpenAI account it expires after a year? I was very annoyed when that happened, no refund no warning it’s just gone as if it was a promo credit. Openrouter is very nice since it puts a barrier between you and those suppliers that were supposed to be like utilities. I got the feeling that if OpenAI was left alone they would be nice as a telco.
- scrollop 5mo agoThough you pay 5% fees? Not worth it for me with the volume of tokens used.
- Futurmix 5mo ago[flagged]
- michaelbuckbee 5mo agoIt's not just comparing all the models, it's also comparing all the providers and configurations of those models. If you're doing any kind of production AI work you'll end up with outages caused by calling a single provider, OpenRouter seamlessly switching between providers is a godsend for uptime. But even more than that there's meaningful cost+speed differences. Here's Sonnet 4.6 being served direct, via Amazon and via Google https://la9q13gg8w.evvl.io/ https://la9q13gg8w.evvl.io/ (spoiler: Google was both fastest and cheapest)