4 ms·
I was surprised to see in the paper that they appear to ask for an answer to a multiple-choice question, and only after getting the answer to the multiple-choic
by ttul 3y ago
I was surprised to see in the paper that they appear to ask for an answer to a multiple-choice question, and only after getting the answer to the multiple-choice question do they ask the model for an explanation. The model will do much better if you ask it to explain its reasoning before it produces its answer to the multiple-choice question.
The reason this works is it puts the reasoning into the context window. Transformers pay attention to everything in the context window and having a rationale sitting there - even if the model generates the rationale - helps the model do a better job of issuing the correct multiple choice answer.
- ahmedfromtunis 3y agoVery interesting. It's like asking the model to have an inner voice in its "mind" and to listen to what that inner voice is telling it.
- baq 3y agoOr like giving the machine time and tape to perform computation. Never tried that but an interesting experiment would be making it output irrelevant things - it could also help to get the right answer just due to the time and space.