4 ms·

Bring up passage that supports your claim. I'll wait.

nl 9mo ago
Not exactly sure what you are looking for here. That GRPO works? > Group Relative Policy Optimization (GRPO), a variant reinforcement learning (RL) algorithm of Proximal Policy Optimization (PPO) (Schulman et al., 2017). GRPO foregoes the critic model, instead estimating the baseline from group scores, significantly reducing training resources. By solely using a subset of English instruction tuning data, GRPO obtains a substantial improvement over the strong DeepSeekMath-Instruct, including both in-domain (GSM8K: 82.9% → 88.2%, MATH: 46.8% → 51.7%) and out-of-domain mathematical tasks (e.g., CMATH: 84.6% → 88.8%) during the reinforcement learning phase Page 2 of https://arxiv.org/pdf/2402.03300 https://arxiv.org/pdf/2402.03300 That GRPO on code works? > Similarly, for code competition prompts, a compiler can be utilized to evaluate the model’s responses against a suite of predefined test cases, thereby generating objective feedback on correctness Page 4 of https://arxiv.org/pdf/2501.12948 https://arxiv.org/pdf/2501.12948
- measurablefunc 9mo ago
  None of those are novel domains w/ their own novel syntax & semantic validators, not to mention the dearth of readily available sources of examples for sampling the baselines. So again, where does it say it works for a programming language with nothing but a grammar & a compiler?
  - nl 9mo ago
    To quote you: > here is no RL for programming languages. and > Either RL works & you have evidence This is just so completely wrong, and here is the evidence. I think everyone in this thread is just surprised you don't seem to know this. Haven't you seen the hundreds of job ads for people to write code for LLMs to train on?
    - measurablefunc 9mo ago
      You're not going to get less confused by doubling down. None of your claims are valid & this is because you haven't actually tried to do what you're suggesting. Taking a grammar & compiler & RLing will get you nowhere.

nl 9mo ago

Not exactly sure what you are looking for here. That GRPO works? > Group Relative Policy Optimization (GRPO), a variant reinforcement learning (RL) algorithm of Proximal Policy Optimization (PPO) (Schulman et al., 2017). GRPO foregoes the critic model, instead estimating the baseline from group scores, significantly reducing training resources. By solely using a subset of English instruction tuning data, GRPO obtains a substantial improvement over the strong DeepSeekMath-Instruct, including both in-domain (GSM8K: 82.9% → 88.2%, MATH: 46.8% → 51.7%) and out-of-domain mathematical tasks (e.g., CMATH: 84.6% → 88.8%) during the reinforcement learning phase Page 2 of https://arxiv.org/pdf/2402.03300 https://arxiv.org/pdf/2402.03300 That GRPO on code works? > Similarly, for code competition prompts, a compiler can be utilized to evaluate the model’s responses against a suite of predefined test cases, thereby generating objective feedback on correctness Page 4 of https://arxiv.org/pdf/2501.12948 https://arxiv.org/pdf/2501.12948

measurablefunc 9mo ago
None of those are novel domains w/ their own novel syntax & semantic validators, not to mention the dearth of readily available sources of examples for sampling the baselines. So again, where does it say it works for a programming language with nothing but a grammar & a compiler?
- nl 9mo ago
  To quote you: > here is no RL for programming languages. and > Either RL works & you have evidence This is just so completely wrong, and here is the evidence. I think everyone in this thread is just surprised you don't seem to know this. Haven't you seen the hundreds of job ads for people to write code for LLMs to train on?
  - measurablefunc 9mo ago
    You're not going to get less confused by doubling down. None of your claims are valid & this is because you haven't actually tried to do what you're suggesting. Taking a grammar & compiler & RLing will get you nowhere.