3 ms·
IIUC, CoT is "incorporated" into training by just providing better quality training data which steers the model towards "thinking" more deeply in its responses.
by bavell 1y ago
IIUC, CoT is "incorporated" into training by just providing better quality training data which steers the model towards "thinking" more deeply in its responses. But at the end of the day, it's still just regular pre training.
RAG - Retrieval augmented generation - how can the retrieval be done during training? RAG will always remain external to the model. The whole point is that you can augment the model by injecting relevant context into the prompt at inference time, bringing your own proprietary/domain-specific data.
- bsenftner 1y agoWho says "during training"? RAG could be built into the functionality of the LLM directly - give it the documents you want it to incorporate, and it ingests them as a temp mini-fine tune. That would work just fine.
- impossiblefork 1y agoThese things with <think> and </think> tokens are actually trained using RL, so it's not like GSM8k or something like that where you just train on some reasoning. It's actually like QuietSTaR but with a focus on a big thought in the beginning and with more sophisticated RL than just REINFORCE (QuietSTaR uses REINFORCE).