3 ms·
I fail to understand how two LLMs would be "consuming" a different amount of tokens given the same input? Does it refer to the number of output tokens? Or is it
by grav 9mo ago
I fail to understand how two LLMs would be "consuming" a different amount of tokens given the same input? Does it refer to the number of output tokens? Or is it in the context of some "agentic loop" (eg Claude Code)?
- bsamuels 9mo agothinking tokens, output tokens, etc. Being more clever about file reads/tool calling.
- jcims 9mo agoOne very specific and limited example, when asked to build something 4.6 seems to do more web searches in the domain to gather latest best practices for various components/features before planning/implementing.
- andrewchilds 9mo agoI've found that Opus 4.6 is happy to read a significant amount of the codebase in preparation to do something, whereas Opus 4.5 tends to be much more efficient and targeted about pulling in relevant context.
- OtomotO 9mo agoAnd way faster too!
- lemonfever 9mo agoMost LLMs output a whole bunch of tokens to help them reason through a problem, often called chain of thought, before giving the actual response. This has been shown to improve performance a lot but uses a lot of tokens
- zozbot234 9mo agoYup, they all need to do this in case you're asking them a really hard question like: "I really need to get my car washed, the car wash place is only 50 meters away, should I drive there or walk?"
- Gracana 9mo agoThey're talking about output consuming from the pool of tokens allowed by the subscription plan.