3 ms·
The closed nature is one thing, but the opaque billing on reasoning tokens is the real dealbreaker for integration. If you are bootstrapping a service, I don't
by storystarling 9mo ago
The closed nature is one thing, but the opaque billing on reasoning tokens is the real dealbreaker for integration. If you are bootstrapping a service, I don't see how you can model your margins when the API decides arbitrarily how long to think and bill for a prompt. It makes unit economics impossible to predict.
- zozbot234 9mo agoYou just have to plan for the worst case.
- TobTobXX 9mo agoDoesn't ClosedAI do the same? Thinking models bill tokens, but the thinking steps are encrypted.
- Rastonbury 9mo agoDestroying unit economics is a bit dramatic... you can chose thinking effort for modern models/APIs and add guidance to the system prompts
- czl 9mo agoFYI: Newer LLM hosting APIs offer control over amount of "thinking" (as well as length of reply) -- some by token count others by an enum (high low, medium, etc.).