5 ms·

> the idea that I can run a complex prompt and have key details of how that prompt was evaluated hidden from me feels like a big step backwards. As a developer

> the idea that I can run a complex prompt and have key details of how that prompt was evaluated hidden from me feels like a big step backwards. As a developer, this is highly concerning, as it makes it much harder to debug where/how the “reasoning” went wrong. The pricing is also silly, because I’m paying for tokens I can’t see. As a user, I don’t really care. LLMs are already magic boxes and I usually only care about the end result, not the path to get there. It will be interesting to see how this progresses, both at OpenAI and other foundation model builders.

bamboozled 2y ago
"trust us, we're using your tokens as efficiently as possible"
OutOfHere 2y ago
> As a user, I don’t really care. As a user, whether of ChatGPT or of the API, I absolutely do care, so I can modify and tune my prompt with the necessary clarifications. My suspicion is that the reason for hiding the reasoning tokens is to prevent other companies from creating a big CoT reasoning dataset using o1. It is anti-competitive behavior. If a user is paying through the nose for the reasoning tokens, and yes they are, the user deserves to be able to see them.
- famouswaffles 2y ago
  >My suspicion is that the reason for hiding the reasoning tokens is to prevent other companies from creating a big CoT reasoning dataset using o1. I mean...they say as much
  - echelon 2y ago
    Once again true to their name.
echelon 2y ago
More opportunity for competitors to differentiate. OpenAI doesn't really have a moat. This isn't payments or SMS where only Stripe or Twilio were trying to win the market. Everybody and their brother is trying to build an LLM business. Grab some researchers, put some compute dollars in, and out comes a product. Everyone wants this market. It's absurdly good for buyers.
deleted 2y ago
[deleted]
OutOfHere 2y ago
> As a user, I don’t really care. Tell me: Just how is it fair for a user to pay for the reasoning tokens without actually seeing them? If they are not shared, the service can bill you anything they want for them!
- ttul 2y ago
  OpenAI’s answer to this would be, “Okay then, don’t use it.”
- mediaman 2y ago
  The simple answer is: I don't care. I'll statistically figure out what the typical total cost per call is from experience, and that's what matters. Who cares if they lie about it, if the model's cost per call fits my budget? If it starts costing $1 per call, and that's too high, then I just won't use it commercially. Whether it was $1 because they inflated the token count or because it just actually took a lot of tokens to do its reasoning isn't really material to my economic decision.
  - lucianbr 2y ago
    The thing is it might increase in cost after you've decided to use it commercially, and have invested a lot of time and resources in it. Now it's very hard to move to something else, but very easy for OpenAI to increase your cost arbitrarily. The statistics you made are not binding for them.
    - nsokolsky 2y ago
      Same applies to every other API in the world, yes.
      - lucianbr 2y ago
        No, S3 pricing for example is predictable, and written in a contract. There's no way for AWS to charge you 3x amount of dollars for 1GB tomorrow. They need to announce it in advance, and give you time to exit the contract if you disagree with the new price. It's really not the same. OpenAI can just tell you your prompt from tomorrow used up 20x times reasoning tokens. There's no advance warning or predictability. I really don't understand how you can claim the situations are identical.
deleted 2y ago
[deleted]
mFixman 2y ago
> As a user, I don’t really care. People should understand and be able to tinker with the tools they use. The tragedy of personal computing is that everything is so abstracted away that users use only a fraction of the power of their computer. People who grew up with modern PCs don't understand the concept of memory, and younger people who grew up with cellphones don't understand the concept of files and directories. Open-weight AI models are great because they let normal users learn how they can make the model work for their particular use cases.

bamboozled 2y ago

"trust us, we're using your tokens as efficiently as possible"

OutOfHere 2y ago

> As a user, I don’t really care. As a user, whether of ChatGPT or of the API, I absolutely do care, so I can modify and tune my prompt with the necessary clarifications. My suspicion is that the reason for hiding the reasoning tokens is to prevent other companies from creating a big CoT reasoning dataset using o1. It is anti-competitive behavior. If a user is paying through the nose for the reasoning tokens, and yes they are, the user deserves to be able to see them.

famouswaffles 2y ago
>My suspicion is that the reason for hiding the reasoning tokens is to prevent other companies from creating a big CoT reasoning dataset using o1. I mean...they say as much
- echelon 2y ago
  Once again true to their name.

echelon 2y ago

More opportunity for competitors to differentiate. OpenAI doesn't really have a moat. This isn't payments or SMS where only Stripe or Twilio were trying to win the market. Everybody and their brother is trying to build an LLM business. Grab some researchers, put some compute dollars in, and out comes a product. Everyone wants this market. It's absurdly good for buyers.

deleted 2y ago

[deleted]

OutOfHere 2y ago

> As a user, I don’t really care. Tell me: Just how is it fair for a user to pay for the reasoning tokens without actually seeing them? If they are not shared, the service can bill you anything they want for them!

ttul 2y ago
OpenAI’s answer to this would be, “Okay then, don’t use it.”
mediaman 2y ago
The simple answer is: I don't care. I'll statistically figure out what the typical total cost per call is from experience, and that's what matters. Who cares if they lie about it, if the model's cost per call fits my budget? If it starts costing $1 per call, and that's too high, then I just won't use it commercially. Whether it was $1 because they inflated the token count or because it just actually took a lot of tokens to do its reasoning isn't really material to my economic decision.
- lucianbr 2y ago
  The thing is it might increase in cost after you've decided to use it commercially, and have invested a lot of time and resources in it. Now it's very hard to move to something else, but very easy for OpenAI to increase your cost arbitrarily. The statistics you made are not binding for them.
  - nsokolsky 2y ago
    Same applies to every other API in the world, yes.
    - lucianbr 2y ago
      No, S3 pricing for example is predictable, and written in a contract. There's no way for AWS to charge you 3x amount of dollars for 1GB tomorrow. They need to announce it in advance, and give you time to exit the contract if you disagree with the new price. It's really not the same. OpenAI can just tell you your prompt from tomorrow used up 20x times reasoning tokens. There's no advance warning or predictability. I really don't understand how you can claim the situations are identical.

deleted 2y ago

[deleted]

mFixman 2y ago

> As a user, I don’t really care. People should understand and be able to tinker with the tools they use. The tragedy of personal computing is that everything is so abstracted away that users use only a fraction of the power of their computer. People who grew up with modern PCs don't understand the concept of memory, and younger people who grew up with cellphones don't understand the concept of files and directories. Open-weight AI models are great because they let normal users learn how they can make the model work for their particular use cases.