3 ms·

I have a question For people in the field! Would GPT only with pretraining and no fine tuning be able to behave better when prompt is "let’s think step by step

by skyde 3y ago

I have a question For people in the field! Would GPT only with pretraining and no fine tuning be able to behave better when prompt is "let’s think step by step"? Or this prompt only worked because a fine tuning dataset containing many "let’s think step by step" prompt was used?

selfhoster11 3y ago
Working with pure pre-trained models is quite hard, and takes some practice. The key part is that "let's think step by step" is a technique that is used by humans also, and therefore (I think) would be somewhat represented in the pre-training corpus. It would be somewhat harder to "activate" this mode of thinking than a "let's think step by step" would in a fine-tuned model, but it would be possible with some elbow grease.