4 ms·
The reasoning modes are really weird with 4.7 In my tests, asking for "none" reasoning resulted in higher costs than asking for "medium" reasoning... Also, "m
by XCSme 6mo ago
The reasoning modes are really weird with 4.7
In my tests, asking for "none" reasoning resulted in higher costs than asking for "medium" reasoning...
Also, "medium" reasoning only had 1/10 of the reasoning tokens 4.6 used to have.
- XCSme 6mo agoOh, and also, the "none" and "medium" variants performed the same (??)
- nisarg2 6mo agoInsane! Even Haiku doesn't make such mistakes.
- XCSme 6mo agoI am not sure it's a mistake, this might be their new "adaptive reasoning" + hidden reasoning trace, so we can't verify.
- amelius 6mo agoClaude is known for its shitty metering.
- Eifert 6mo agoMedium reasoning has regressed since 4.6. While None and Max have improved since 4.6 in our benchmark. We suspect that this is how Claude tries to cope with the increased user base. Note, Google and OpenAI probably did something similar long ago.