4 ms·
The latter. A reasoning model has been finetuned to use the scratchpad for intermediate results (which works better than just prompting a model to do the same).
by rcxdude 1y ago
The latter. A reasoning model has been finetuned to use the scratchpad for intermediate results (which works better than just prompting a model to do the same).
- NiloCK 1y agoI'd expect the same (fine tuning to be better than mere prompting) for most anything. So a model is or is not "a reasoning model" according to the extent of a fine tune. Are there specific benchmarks that compare models vs themselves with and without scratchpads? High with:without ratios being reasonier models? Curious also how much a generalist model's one-shot responses degrade with reasoning post-training.
- bigyabai 1y ago> Are there specific benchmarks that compare models vs themselves with and without scratchpads? Yep, it's pretty common for many models to release an instruction-tuned and thinking-tuned model and then bench them against each other. For instance, if you scroll down to "Pure text performance" there's a comparison of these two Qwen models' performance: https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Thinking https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Thinking
- walthamstow 1y agoThanks for the Qwen tip. Interesting how much of a difference reasoning makes for coding.
- dcre 1y agoThe question is: fine-tuning for what? Reasoning is not a particular task, it is a general-purpose technique for directing more compute at any task.
- irthomasthomas 1y agoPivot tokens like 'wait', 'actually' and 'alternatively' are boosted in order to force the model to explore alternate solutions.
- robkop 1y ago> Are there specific benchmarks that compare models vs themselves with and without scratchpads? High with:without ratios being reasonier models? Yes, simplest example: https://www.anthropic.com/engineering/claude-think-tool https://www.anthropic.com/engineering/claude-think-tool