4 ms·

One question I was wondering about regarding the open models released by big labs is how much more the could improve with additional training. GPT-OSS has 2.1m

by oezi 1y ago

One question I was wondering about regarding the open models released by big labs is how much more the could improve with additional training. GPT-OSS has 2.1m hours of training, how much score improvements could we see at double that?

poorman 1y ago
As we saw with GPT-5 the RL technique of training doesn't scale forever
- oezi 1y ago
  I meant scaling the base training before RL.
- energy123 1y ago
  Unless GPT-5 is 30% cheaper to run than o3. Then it's scaling brilliantly given the small gap between release dates. People are really drawing too many conclusions from too little information.
ModelForge 1y ago
I think GPT-4.5 was potentially the original GPT-5 model that was larger and pre-trained on more data. Too bad it was too expensive to deploy at scale so that we never saw the RL-ed version