> why wasn't this the immediate next step that everyone took. It was actually tested by various labs. Just probably not at this scale. The first model that fe

> why wasn't this the immediate next step that everyone took. It was actually tested by various labs. Just probably not at this scale. The first model that featured RL prominently was DeepSeek-math-7b-RL, published last year in april. It was at the time the best model for math, and remained so until the qwen2.5-math series, that probably had way more data put into them. There's a thing about RL that makes it tricky - the models tend to behave very stubbornly. That is, if they see something that resembles their training method (i.e. math problems), they'll solve the problem, and they'll be good at it. But if you want something close to that but not quite solving it (i.e. analyse this math problem and write hints, or here are 5 problems extract the common methods used for solving, etc.) you'll see that they perform very poorly, often times just going straight into "to solve this problem we...". This is even mentioned in the R1 paper. Poor adherence to prompts, especially ssytem prompts. So that is still challenging.