3 ms·
If pre-training is just training, then how on earth can OpenAI not have "a successful pre-training run"? The word successful indicates that they tried, but fail
by mrweasel 11mo ago
If pre-training is just training, then how on earth can OpenAI not have "a successful pre-training run"? The word successful indicates that they tried, but failed.
It might be me misunderstanding how this works, but I assumed that the training phase was fairly reproducible. You might get different results on each run, do to changes in the input, but not massively so. If OpenAI can't continuously and reliably train new models, then they are even more overvalued that I previously assumed.
- immibis 11mo agoMaybe this has something to do with why they're declaring "code red".
- nodja 11mo agoBecause success for them doesn't mean it works, it means it works much better than what they currently have. If a 1% improvement comes at the cost of spending 10x more on training and 2x more on inference then you're failing at runs. (numbers out of ass)
- mrweasel 11mo agoThat makes sense. It's not that the training didn't complete or returned a moronic model, but the capabilities have plateaued.