3 ms·
Recent paper from Anthropic on the fact that the reasoning output does not necessarily reflect what the LLM is "thinking". https://arxiv.org/abs/2305.04388 htt
by oersted 2y ago
Recent paper from Anthropic on the fact that the reasoning output does not necessarily reflect what the LLM is "thinking".
https://arxiv.org/abs/2305.04388 https://arxiv.org/abs/2305.04388
That being said, your idea is not unreasonable. The way DeepSeek phrased it, it just sounds like implementing such solutions might be a hassle greatly increasing complexity, and they were just focused on making an RL baseline work at scale.
- devmor 2y agoI was actually thinking of that paper when I wrote that comment, hence the frustration that we don't actually know the intermediates. Still, perhaps the stepped output we get may hint at that kind of "cheating" and can be used in reinforcement... or perhaps that kind of reinforcement will just make the LLMs better at cheating. The problem is definitely a lot more complex than the trivial way I referenced it, at least.