4 ms·
No, the CoT is not simply extra context the models are specifically trained to use CoT and that includes treating it as unspoken thought
by wahnfrieden 2y ago
No, the CoT is not simply extra context the models are specifically trained to use CoT and that includes treating it as unspoken thought
- reaperman 2y agoHuge thank you for correcting me. Do you have any good resources I could look at to learn how the previous CoT is included in the input tokens and treated differently?
- wahnfrieden 2y agoI've only read the marketing materials of closed models. So they could be lying, too. But I don't think CoT is something you can do with pre-CoT models via prompting and context manipulation. You can do something that looks a little like CoT, but the model won't have been trained specifically on how to make good use of it and will treat it like Q&A context.