3 ms·
My immediate question: How does this compare to simply maximizing the average probability (or log-probability) of the top-k paths predicted by the model?
by cs702 3y ago
My immediate question:
How does this compare to simply maximizing the average probability (or log-probability) of the top-k paths predicted by the model?
- evmar 3y agoThey discuss this: "Table 1 illustrates that CoT paths do not consistently outrank non-CoT ones in the model’s probability assessment." and then look at table 1 to see some examples. E.g. in the first one the top 8 paths are wrong and it's the 9th that is correct.
- cs702 3y ago> They discuss this Based on my skim of the paper, they don't. They select the top k paths based on the probability of the first token only.