3 ms·
> Since it looks like reasoning, the default position to be disproved is this is reasoning. Since we know it is a model that is trained to generate text that h
by Jensson 2y ago
> Since it looks like reasoning, the default position to be disproved is this is reasoning.
Since we know it is a model that is trained to generate text that humans would generate, it writes down not its reasoning but what it thinks a human would write in that scenario.
So it doesn't write its reasoning there, if it does reason its behind the words and not the words itself.
- ludwik 2y agoSure, but we have clear evidence that generating this pseudo-reasoning text helps the model to make better decisions afterwards. Which means that it not only looks like reasoning but also effectively serves the same purpose. Additionally, the new "reasoning" models don't just train on human text - they also undergo a Reinforcement Learning training step, where they are trained to produce whatever kinds of "reasoning" text help them "reason" best (i.e., leading to correct decisions based on that reasoning). This further complicates things and makes it harder to say "this is one thing and one thing only".