3 ms·
Testing at max effort likely doesn't produce optimal results.
by adgjlsfhk1 2mo ago
Testing at max effort likely doesn't produce optimal results.
- sggyamg 2mo agoCan you be more explicit?
- adgjlsfhk1 2mo agoMax effort is the way to give the highest perf, but not highest perf/$. Having claude (or other models) use a lower effort can often be 80% as smart but get to the results 10x faster for the problems where it works.
- crimist 2mo agoWe've seen models perform worse at higher efforts in our vuln detection evals. For example IIRC gpt 5.5 and 5.6 both scored better or high as compared to xhigh.