3 ms·
RL is barely even a training method, its more of a dataset generation method.
by whatshisface 11mo ago
RL is barely even a training method, its more of a dataset generation method.
- theOGognf 11mo agoI feel like both this comment and the parent comment highlight how RL has been going through a cycle of misunderstanding recently from another one of its popularity booms due to being used to train LLMs
- mistercheph 11mo agocare to correct the misunderstanding?
- mountainriver 11mo agoI mean DPO, PPO, and GRPO all use losses that are not what’s used with SFT for one. They also force exploration as a part of the algorithm. They can be used for synthetic data generation once the reward model is good enough.
- phyalow 11mo agoIts reductive, but also roughly correct.
- singularity2001 11mo agoWhile collecting data according to policy is part of RL, 'reductive' is an understatement. It's like saying algebra is all about scalar products. Well yes, 1%