3 ms·

Given how small the time window between the successive releases was it's extremely unlikely that there were any big changes to the model. Most likely it's just

Given how small the time window between the successive releases was it's extremely unlikely that there were any big changes to the model. Most likely it's just better preprocessed training data, more training data, trained for longer, performance optimizations for attention, or a few changes to layer sizes.

mach1ne 4y ago
Your timeline is wrong, GPT-4 finished training already in August.
Veen 4y ago
They didn’t release GTP-4 immediately after it was trained and then move on to training GPT-5. They had 4 for almost 6 months before it was released. 5 was certainly well underway long before we’d heard of 4.