4 ms·
You're right about the original GRPO proposal, but there are simplified variants that do just use best of K sampling. GRPO (or GRPO like approaches) for diffus
by zimablue22 2mo ago
You're right about the original GRPO proposal, but there are simplified variants that do just use best of K sampling.
GRPO (or GRPO like approaches) for diffusion/flow matching similarly can be likelihood free.
- yorwba 2mo agoWithout anchoring on the group average it's hardly Group-Relative and without likelihood there's no Policy; that doesn't leave much of the original GRPO unchanged... Do you have a particular paper in mind that does likelihood-free best-of-K but just calls it GRPO?
- zimablue22 2mo agoAny of the Best of N papers that exploded in popularity after GRPO. Likelihood is not fundamental to the spirit of GRPO, any exploratory mechanism would work. That sequential LLMs have a step-wise probability is convenient but not critical to this approach (where rejection sampling is widely used in diffusion models).
- kamranjon 2mo agoDo you have an example? Would love to read one.