4 ms·
by the love of god, please stop overfitting on gsm8k
by sadboots 1y ago
by the love of god, please stop overfitting on gsm8k
- i5heu 1y agoIt looks like your neural network is overfitted on seeing overfitt where is none. Prejudices is a form of overfitting IMHO
- t55 1y agoagree, the RG evals feel like a fresh breeze
- olliestanley 1y agoDifficult one. GSM8K and MATH evals (both reported in Reasoning Gym paper) are common in smaller model RL papers for a reason, which is that smaller models can get decent scores on them, unlike fresher & harder benchmarks. Part of the aim of RG is to be used as a difficulty-adjustable & non-repeating eval though so if people think it's a good benchmark, perhaps it will allow this status quo to shift!