3 ms·

I guess I'm bearish? It's not that they trained a new model, but they took an existing model and RL'd it a bit? The scores are very close to QwQ-32B, and at t

I guess I'm bearish? It's not that they trained a new model, but they took an existing model and RL'd it a bit? The scores are very close to QwQ-32B, and at the end: "Overall, as QwQ-32B was already extensively trained with RL, it was difficult to obtain huge amounts of generalized improvement on benchmarks beyond our improvements on the training dataset. To see stronger improvements, it is likely that better base models such as the now available Qwen3, or higher quality datasets and RL environments are needed."

fabmilo 1y ago
The interesting delta here is that this proves that we can distribute the training and get a functioning model. The scaling factor is way bigger than datacenters
- refulgentis 1y ago
  The RL, not the training. No?
  - itchyjunk 1y ago
    RL is still training. Just like pretraining is still training. SFT is also training. This is how I look at it. Models weights are being updated in all cases.
    - refulgentis 1y ago
      Simplifying it down to "adjusting any weights is training, ipso facto this is meaningful" obscures more light than it sheds (as they noted, RL doesn't get you very far, at all)
- comex 1y ago
  But does that mean much when the training that produced the original model was not distributed?
- deleted 1y ago
  [deleted]
christianqchung 1y ago
Third party fine tuned open weighted LLMs tend to be good at a handful of benchmarks, but parity or lower on others compared to the original model. There are some exceptions like Nvidia's Nemotron series, but the differences generally are so small as to be imperceptible. Deepseek released finetunes of several Qwen and Llama models alongside R1, and while they were better in some select (mostly math) and coding domains, there were problems resulting from fine tuning that didn't result in them overtaking the original models in usage.
cess11 1y ago
Seems that's mostly a byproduct from working on the core business idea, GPU arbitrage.

fabmilo 1y ago

The interesting delta here is that this proves that we can distribute the training and get a functioning model. The scaling factor is way bigger than datacenters

refulgentis 1y ago
The RL, not the training. No?
- itchyjunk 1y ago
  RL is still training. Just like pretraining is still training. SFT is also training. This is how I look at it. Models weights are being updated in all cases.
  - refulgentis 1y ago
    Simplifying it down to "adjusting any weights is training, ipso facto this is meaningful" obscures more light than it sheds (as they noted, RL doesn't get you very far, at all)
comex 1y ago
But does that mean much when the training that produced the original model was not distributed?
deleted 1y ago
[deleted]

christianqchung 1y ago

Third party fine tuned open weighted LLMs tend to be good at a handful of benchmarks, but parity or lower on others compared to the original model. There are some exceptions like Nvidia's Nemotron series, but the differences generally are so small as to be imperceptible. Deepseek released finetunes of several Qwen and Llama models alongside R1, and while they were better in some select (mostly math) and coding domains, there were problems resulting from fine tuning that didn't result in them overtaking the original models in usage.

cess11 1y ago

Seems that's mostly a byproduct from working on the core business idea, GPU arbitrage.