2 ms·
Doesn’t this assume that the probability of a correct answer is iid? It can’t be that simple.
by ziofill 2y ago
Doesn’t this assume that the probability of a correct answer is iid? It can’t be that simple.
- vbarrielle 2y agoYes the main flaw of this reasoning is supposing that e does not depend on previous output. I think this was a good approximation to characterize vanilla LLMs, but the kind of RL in this paper is done with the explicit goal of making e depending on prior output (and specifically to lower it given a long enough chain of thought).