4 ms·

Here are some things that are abundantly clear to me. 1.) LLMs are useful for programming 2.) Open models are excellent and will continue to improve 3.) Econ

Here are some things that are abundantly clear to me. 1.) LLMs are useful for programming 2.) Open models are excellent and will continue to improve 3.) Economies of scale and ease of access mean self hosting is out of the question for a large number of users This means that even if the largest labs are not worth trillions and a large amount of the data center build out is not as valuable as the builders project and GPU/RAM prices plummet, it will not matter at all for this business. People will want to buy cheap open source tokens from a centralized trusted provider. $7 billion for a business with little overhead that is already within their core competency and has strategic growth potential seems like a very good deal.

27183 2mo ago
Unfortunately we don't really know anything about OpenRouter or Stripe's numbers. They're both private companies. AFAIK there really isn't much information publicly available about how profitable or not OpenRouter actually is, or whether it has a little or a lot of overhead. So I can't really say whether $7B is reasonable or inflated. It seems like an enormous number, and my hunch is like all the other enormous numbers floating around in the AI space it's probably inflated by the hype bubble. But we just don't know. Broadly I agree with you. It seems like they have an in-demand product and there could be a sustainable business there at least in principle. But whether it's a $7B business or a $70M business I can't say.
blobbers 2mo ago
What does openrouter actually do though? They buy a bunch of tokens from token providers and allow you to easily route queries between token processors? This seems like the kind of thing that you can have Claude write in an afternoon for whatever service you're running. I don't see the value.
- byzantinegene 2mo ago
  i think plenty of people don't want to be forced to using just Claude and ChatGPT
- sipjca 2mo ago
  people are lazy people don't want to maintain infra (ie adding new models all the time) people want their queries to work without thinking if a provider goes down, openrouter queries dont (ideally) it's really not that complex to understand
  - blobbers 2mo ago
    Yeah, but who wants to deploy a model just like that. I mean, what if that model doesn't handle the same guard rails. There's a million reasons why switching models should not be a snap decision. The back-up model is not a bad idea, but nonetheless...
- skybrian 2mo ago
  My understanding is that it’s a common LLM API with common billing for multiple LLM providers. You can switch between LLM providers whenever you like, and the way you pay for API usage doesn’t change. You don’t even need to create an account with the LLM provider. I don’t use OpenRouter, but exe.dev has a similar feature. I choose which LLM to use in a pulldown menu. But the downside of this is that you’re paying API prices, which are much higher than subscription prices. So, recently they added a way to connect a ChatGPT subscription and I mostly use that. But I can try out other models any time. So this doesn’t seem like all that much of a moat to me, but it’s still a convenient service to have, since you aren’t going to set up billing with all the LLM providers in an afternoon. I expect there will be multiple competitive businesses. Compare with what distributors do for physical goods.
enos_feedler 2mo ago
Why a centralized trusted provider? Linux VMs are also useful (host services that are the end result of programming) and open source and fit for cloud convenience. I dont go to OpenLinuxVM to get my vms. I just get them from whereever. Aws. Digital ocean. You name it. Why centralize tokens?
- LPisGood 2mo ago
  You go to AWS or Azure, etc. These may as well be called OpenLinuxVM
  - enos_feedler 2mo ago
    Thats my point. There isn't a single centralized aggregator and there is very little value in it. Most of the margins come from other kinds of APIs. Gcp, azure and aws are not differentiated on openlinuxvm its just race to the bottom pricing

27183 2mo ago

Unfortunately we don't really know anything about OpenRouter or Stripe's numbers. They're both private companies. AFAIK there really isn't much information publicly available about how profitable or not OpenRouter actually is, or whether it has a little or a lot of overhead. So I can't really say whether $7B is reasonable or inflated. It seems like an enormous number, and my hunch is like all the other enormous numbers floating around in the AI space it's probably inflated by the hype bubble. But we just don't know. Broadly I agree with you. It seems like they have an in-demand product and there could be a sustainable business there at least in principle. But whether it's a $7B business or a $70M business I can't say.

blobbers 2mo ago

What does openrouter actually do though? They buy a bunch of tokens from token providers and allow you to easily route queries between token processors? This seems like the kind of thing that you can have Claude write in an afternoon for whatever service you're running. I don't see the value.

byzantinegene 2mo ago
i think plenty of people don't want to be forced to using just Claude and ChatGPT
sipjca 2mo ago
people are lazy people don't want to maintain infra (ie adding new models all the time) people want their queries to work without thinking if a provider goes down, openrouter queries dont (ideally) it's really not that complex to understand
- blobbers 2mo ago
  Yeah, but who wants to deploy a model just like that. I mean, what if that model doesn't handle the same guard rails. There's a million reasons why switching models should not be a snap decision. The back-up model is not a bad idea, but nonetheless...
skybrian 2mo ago
My understanding is that it’s a common LLM API with common billing for multiple LLM providers. You can switch between LLM providers whenever you like, and the way you pay for API usage doesn’t change. You don’t even need to create an account with the LLM provider. I don’t use OpenRouter, but exe.dev has a similar feature. I choose which LLM to use in a pulldown menu. But the downside of this is that you’re paying API prices, which are much higher than subscription prices. So, recently they added a way to connect a ChatGPT subscription and I mostly use that. But I can try out other models any time. So this doesn’t seem like all that much of a moat to me, but it’s still a convenient service to have, since you aren’t going to set up billing with all the LLM providers in an afternoon. I expect there will be multiple competitive businesses. Compare with what distributors do for physical goods.

enos_feedler 2mo ago

Why a centralized trusted provider? Linux VMs are also useful (host services that are the end result of programming) and open source and fit for cloud convenience. I dont go to OpenLinuxVM to get my vms. I just get them from whereever. Aws. Digital ocean. You name it. Why centralize tokens?

LPisGood 2mo ago
You go to AWS or Azure, etc. These may as well be called OpenLinuxVM
- enos_feedler 2mo ago
  Thats my point. There isn't a single centralized aggregator and there is very little value in it. Most of the margins come from other kinds of APIs. Gcp, azure and aws are not differentiated on openlinuxvm its just race to the bottom pricing