4 ms·

I just want a little predictable insight into how much I get. For example, at a buffet, I know I can only eat so much food and can plan around it. This is like

I just want a little predictable insight into how much I get. For example, at a buffet, I know I can only eat so much food and can plan around it. This is like going to a buffet and not knowing how many plates I can take or how big the plates are, and it changes each week, and yet I have to keep paying the same price. Except it's not about eating, it's about my work and deadlines and promises and all that.

kristjansson 6mo ago
If you need the tokens for real work, that’s what the API and the other providers like Bedrock are for. The subscription product is merely to whet your appetite.
- gowld 6mo ago
  Missing the point. I don't choose which tokens to buy. I send a request and the server decides how much it costs after its done.
- jimkleiber 6mo ago
  Well then I would just not use their service. I used extra usage once and just for what I'd consider a low amount of tests and coding, racked up like $300 in an hour or more. For some, that's not a lot of money, for me, I'd just code it manually, especially without knowing almost any way to gauge how much I'll need and how fast it goes. I'm not sure how businesses budget for llm APIs, as they seem wildly unpredictable to me and super expensive, but maybe I'm missing something about it.
_flux 6mo ago
That's what these providers want as well, but from the other side. They want to know that a customer won't be able to eat more than certain number of servings, as they need to pay for each of those servings. It works out even if some customers are able to eat a lot, because people on average have a certain limit. The limits of computers are much higher.
- jimkleiber 6mo ago
  Fair, and I think openclaw and all the orchestrators are having agents maxing out the plans. So maybe they figure out a new tier that is agent-run vs human-run. Agents are much more insatiable, whereas humans have a limit. Not sure if it'd be possible to split between those two different modes, but I think that might address the appetite issue better.
  - _flux 6mo ago
    I think it would be impossible to find the price point for a monthly subscribtion that would both be profitable to a provider, as well as attractive to the customer. If anything, as the customers would now be paying necessarily even a higher amount for agentic use, they'd make certain they'd be using the agent as effectively as they can, meaning it would be even more costly for the provider. Pay-per-token is really the only way it can work. If some kind of fixed monthly price is desirable, then there should be a quota the user can assign, and then the agent could e.g. slow itself down by 50% when 50% of the quota is spent, another 50% at 75%, etc, to make it last longer.. As a side thought, I wonder how it could affect an agent's behavior if the information of this token usage/limit was brought to it..
    - jimkleiber 6mo ago
      Ironically in chatting with Gemini, helped me realize that telecoms have often solved this problem of unlimited usage with rate/speed limiting. If one goes over 10GB or whatnot of data, then the speed drops from 5G to 3G, and data remains unlimited. I wonder if there could be something like that, maybe even a progressive rate limiting, where after a certain number of tokens or another metric of use, then the speed slows down a LOT. Not saying that I would love that as a consumer, as I'd prefer this all-you-can eat, unlimited data plan, but I wonder if that would be a compromise that could work, as it seems to have worked OK with the telecom space. edit: the nerd in me loves the irony of me making the above comment and then later seeing your username as flux :-)
criddell 6mo ago
When you hire a person, you don't know what you are going to get out of them today. If an hour of an excellent developer's time is worth $X, isn't that the upper bound of what the AI companies can charge? If hiring a person is better value than paying for an AI, then do that.
- jimkleiber 6mo ago
  Fair on not knowing what you'll get out of someone. But if that varies wildly, I may not want to hire that person. Even with employment, predictability matters a lot. If they underperform too much, I might feel annoyed. If they overperform, I might feel guilty. They can charge whatever they want, I think many people like to make business decisions based on relative predictability or at least be more aware that there's a risk. If they want it to be "some weeks you have lots of usage, some weeks less, and it depends on X factors, or even random factors" then people could make a more informed choice. I think now it's basically incredibly vague and that works while it's relatively predictable, and starts to fail when it's not, for those that wanted the implied predictability.

kristjansson 6mo ago

If you need the tokens for real work, that’s what the API and the other providers like Bedrock are for. The subscription product is merely to whet your appetite.

gowld 6mo ago
Missing the point. I don't choose which tokens to buy. I send a request and the server decides how much it costs after its done.
jimkleiber 6mo ago
Well then I would just not use their service. I used extra usage once and just for what I'd consider a low amount of tests and coding, racked up like $300 in an hour or more. For some, that's not a lot of money, for me, I'd just code it manually, especially without knowing almost any way to gauge how much I'll need and how fast it goes. I'm not sure how businesses budget for llm APIs, as they seem wildly unpredictable to me and super expensive, but maybe I'm missing something about it.

_flux 6mo ago

That's what these providers want as well, but from the other side. They want to know that a customer won't be able to eat more than certain number of servings, as they need to pay for each of those servings. It works out even if some customers are able to eat a lot, because people on average have a certain limit. The limits of computers are much higher.

jimkleiber 6mo ago
Fair, and I think openclaw and all the orchestrators are having agents maxing out the plans. So maybe they figure out a new tier that is agent-run vs human-run. Agents are much more insatiable, whereas humans have a limit. Not sure if it'd be possible to split between those two different modes, but I think that might address the appetite issue better.
- _flux 6mo ago
  I think it would be impossible to find the price point for a monthly subscribtion that would both be profitable to a provider, as well as attractive to the customer. If anything, as the customers would now be paying necessarily even a higher amount for agentic use, they'd make certain they'd be using the agent as effectively as they can, meaning it would be even more costly for the provider. Pay-per-token is really the only way it can work. If some kind of fixed monthly price is desirable, then there should be a quota the user can assign, and then the agent could e.g. slow itself down by 50% when 50% of the quota is spent, another 50% at 75%, etc, to make it last longer.. As a side thought, I wonder how it could affect an agent's behavior if the information of this token usage/limit was brought to it..
  - jimkleiber 6mo ago
    Ironically in chatting with Gemini, helped me realize that telecoms have often solved this problem of unlimited usage with rate/speed limiting. If one goes over 10GB or whatnot of data, then the speed drops from 5G to 3G, and data remains unlimited. I wonder if there could be something like that, maybe even a progressive rate limiting, where after a certain number of tokens or another metric of use, then the speed slows down a LOT. Not saying that I would love that as a consumer, as I'd prefer this all-you-can eat, unlimited data plan, but I wonder if that would be a compromise that could work, as it seems to have worked OK with the telecom space. edit: the nerd in me loves the irony of me making the above comment and then later seeing your username as flux :-)

criddell 6mo ago

When you hire a person, you don't know what you are going to get out of them today. If an hour of an excellent developer's time is worth $X, isn't that the upper bound of what the AI companies can charge? If hiring a person is better value than paying for an AI, then do that.

jimkleiber 6mo ago
Fair on not knowing what you'll get out of someone. But if that varies wildly, I may not want to hire that person. Even with employment, predictability matters a lot. If they underperform too much, I might feel annoyed. If they overperform, I might feel guilty. They can charge whatever they want, I think many people like to make business decisions based on relative predictability or at least be more aware that there's a risk. If they want it to be "some weeks you have lots of usage, some weeks less, and it depends on X factors, or even random factors" then people could make a more informed choice. I think now it's basically incredibly vague and that works while it's relatively predictable, and starts to fail when it's not, for those that wanted the implied predictability.