3 ms·
I wonder what Sutton thinks about some of the more recent innovations in RL like GRPO. In some ways it’s new, in other ways it’s an echo of RLOO.
by janalsncm 3mo ago
I wonder what Sutton thinks about some of the more recent innovations in RL like GRPO. In some ways it’s new, in other ways it’s an echo of RLOO.
- porridgeraisin 3mo agoGRPO is policy gradient/PPO with your value function baseline monte carlo estimated using k rollouts. The only new thing is finding out it works well with binary rewards and LLM policies.
- janalsncm 3mo agoIt is a huge improvement to PPO because you don’t need a separate critic model which cuts memory costs in half and stabilizes training.
- porridgeraisin 3mo agoYes, but monte carlo estimating the critic model is not new.