3 ms·
Most important, R1 shut down some very complex ideas (like DPO & MCTS) and showed that the path forward is simple, basic RL. This isn't quite true. R1 used a m
by rahimnathwani 2y ago
Most important, R1 shut down some very complex ideas (like DPO & MCTS) and showed that the path forward is simple, basic RL.
This isn't quite true. R1 used a mix of RL and supervised fine-tuning. The data used for supervised fine-tuning may have been model-generated, but the paper implies it was human-curated: they kept only the 'correct' answers.
- Philpax 2y agoI think what you're saying is consistent with the quote: human curation of SFT data is indeed not complex. There might be extra work on top of that RL, but it's the same work that's been done throughout LLM development.
- tkellogg 2y agoAdditionally, in the following days, I've seen evidence suggesting that the SFT part might not even be necessary. I'd argue that work wouldn't have happened if R1 wasn't released in the open.