3 ms·

This whole business model of trying to shave off or arbitrage a fraction of the money going to OpenAI and Anthropic just sucks. And it seems precarious. There's

This whole business model of trying to shave off or arbitrage a fraction of the money going to OpenAI and Anthropic just sucks. And it seems precarious. There's no honest way to resell tokens at a profit, and everyone knows it.

Havoc 1y ago
>There's no honest way to resell tokens at a profit, and everyone knows it. Agree with the sentiment, but I do think there are edge cases. e.g. I could see a place like openrouter getting away with a tiny fractional markup based on the value they provide in the form of having all providers in one place
- Lalabadie 1y ago
  The issue with a model like this (fixed small percentage) is that your biggest clients are the most incentivized to move away. At scale, OpenRouter will instead get you the lower high-volume fees they themselves get from their different providers.
thelastbender12 1y ago
Sorry, how is this new pricing anything but honest? They provide an editor you can use to - optimize the context you send to the LLM services - interact with the output that comes out of them Why does not justify charging a fraction of your spend on the LLM platform? This is pretty much how every service business operates.
- hu3 1y ago
  There's now greater incentive for Zed to stuff more content in the prompts to inflate tokens used and thus profit more. Or at least be less zealous. This is not a new concern. And is not unique to Zed.
drakythe 1y ago
For companies where that is their entire business model I absolutely agree. Zed is a solid editor with additional LLM integration features though, so this move would seem to me to just cover their costs + some LLM integration development funds. If their users don't want to use the LLM then no skin off Zed's back unless they've signed some guaranteed usage contract.
dinvlad 1y ago
The whole business model even for OAI/Anthropic is unsustainable.. they are already running it at a huge loss atm, and will do for the foreseeable future. The economics simply doesn't work, unfortunately or not
- porridgeraisin 1y ago
  OAI/Anthropic/Big AI players today never ever need to become profitable... The same way roads never need to become profitable by themselves. This per token charge is like tolls, it has nothing to do with the costs. AI, just like cloud was before this, is being treated as "infrastructure". The reason people invest in roads is not to make money from the road itself, but rather recuper all the losses from the new extremely profitable businesses that will come on top of this infrastructure. To the stock ticker, this looks like a boom+bust cycle. But really, it's a bust+boom cycle as far as the investors are concerned. Saudi Arabia don't give a hoot about transformers, but they know if they invest their large amounts of capital in the new infrastructure - huge models in expensive datacenters using cities worth of power, they can invest in the many new hyper profitable businesses on top. Also helps that it locks in demand for oil, I guess :P A similar example for cloud would be the gajillion cloud SaaS, DevOps, cloud security whatever businesses that only exist today because the whole cloud infra segment ran unprofitable for a long time and created the infrastructure for all this. These new businesses will not be multi hundred-billion dollar businesses, no. They will all be million dollar businesses. But you'll have a million such businesses. Everyone that's stuffing money into AI today is hoping that there will be a huge product layer [1] on top of this new infrastructure once it's all consolidated at a few major players allowing "infra" costs to drop considerably, which they can milk. The stock of all these companies going high is just wall st making it easy for people to buy debt from this infrastructure. If they're indisciplined, they will co-mingle this debt with normal people's daily lives debt and sell it as one, and everything gets affected badly. If they're not, then the stock market will complain, but normal folks will be kind of OK. [1] well, everyone has this same idea, meaning there also a lot of short term investors trying to make gains while openai/nvidia/etc are on their way up, sort of like greater fool investing, but let's ignore them for the purposes of this argument. [Note] of course, whether cloud/ai/whatever are actually useful infra that deserves to be force-created is up for debate.. many disagree, not me though.

Havoc 1y ago

>There's no honest way to resell tokens at a profit, and everyone knows it. Agree with the sentiment, but I do think there are edge cases. e.g. I could see a place like openrouter getting away with a tiny fractional markup based on the value they provide in the form of having all providers in one place

Lalabadie 1y ago
The issue with a model like this (fixed small percentage) is that your biggest clients are the most incentivized to move away. At scale, OpenRouter will instead get you the lower high-volume fees they themselves get from their different providers.

thelastbender12 1y ago

Sorry, how is this new pricing anything but honest? They provide an editor you can use to - optimize the context you send to the LLM services - interact with the output that comes out of them Why does not justify charging a fraction of your spend on the LLM platform? This is pretty much how every service business operates.

hu3 1y ago
There's now greater incentive for Zed to stuff more content in the prompts to inflate tokens used and thus profit more. Or at least be less zealous. This is not a new concern. And is not unique to Zed.

drakythe 1y ago

For companies where that is their entire business model I absolutely agree. Zed is a solid editor with additional LLM integration features though, so this move would seem to me to just cover their costs + some LLM integration development funds. If their users don't want to use the LLM then no skin off Zed's back unless they've signed some guaranteed usage contract.

dinvlad 1y ago

The whole business model even for OAI/Anthropic is unsustainable.. they are already running it at a huge loss atm, and will do for the foreseeable future. The economics simply doesn't work, unfortunately or not

porridgeraisin 1y ago
OAI/Anthropic/Big AI players today never ever need to become profitable... The same way roads never need to become profitable by themselves. This per token charge is like tolls, it has nothing to do with the costs. AI, just like cloud was before this, is being treated as "infrastructure". The reason people invest in roads is not to make money from the road itself, but rather recuper all the losses from the new extremely profitable businesses that will come on top of this infrastructure. To the stock ticker, this looks like a boom+bust cycle. But really, it's a bust+boom cycle as far as the investors are concerned. Saudi Arabia don't give a hoot about transformers, but they know if they invest their large amounts of capital in the new infrastructure - huge models in expensive datacenters using cities worth of power, they can invest in the many new hyper profitable businesses on top. Also helps that it locks in demand for oil, I guess :P A similar example for cloud would be the gajillion cloud SaaS, DevOps, cloud security whatever businesses that only exist today because the whole cloud infra segment ran unprofitable for a long time and created the infrastructure for all this. These new businesses will not be multi hundred-billion dollar businesses, no. They will all be million dollar businesses. But you'll have a million such businesses. Everyone that's stuffing money into AI today is hoping that there will be a huge product layer [1] on top of this new infrastructure once it's all consolidated at a few major players allowing "infra" costs to drop considerably, which they can milk. The stock of all these companies going high is just wall st making it easy for people to buy debt from this infrastructure. If they're indisciplined, they will co-mingle this debt with normal people's daily lives debt and sell it as one, and everything gets affected badly. If they're not, then the stock market will complain, but normal folks will be kind of OK. [1] well, everyone has this same idea, meaning there also a lot of short term investors trying to make gains while openai/nvidia/etc are on their way up, sort of like greater fool investing, but let's ignore them for the purposes of this argument. [Note] of course, whether cloud/ai/whatever are actually useful infra that deserves to be force-created is up for debate.. many disagree, not me though.