3 ms·
> And here’s the thing – we all know that GPT-3 was vastly better than GPT-2. And we all know that GPT-4 (released thirteen months ago) was vastly better than G
by thorum 2y ago
> And here’s the thing – we all know that GPT-3 was vastly better than GPT-2. And we all know that GPT-4 (released thirteen months ago) was vastly better than GPT-3. But what has happened since?
Other models have gradually caught up to GPT-4 while OpenAI has declined to discuss or release anything regarding their internal research and work on GPT-5. Given how far ahead they were with GPT-4, it seems odd to assume no further progress has been made and just not published yet.
The author’s assumption is based on the claim that GPT-4 Turbo “was a failed attempt at GPT-5” but I don’t see any good reason to believe this unless you take the claim that GPT-5 is impossible as a given and reason backwards.
- kromem 2y agoAlso, part of the issue with the argument is that smaller firms like Anthropic now have models that exceed OpenAI's offering. If we were actually at a point of diminishing returns, they'd only be catching up with similar levels of investment, not bypassing at lower levels. And when digging into the details of the differences between something like Opus and GPT-4, it becomes more readily apparent why OpenAI's product line up is struggling. Their current issues (to the extent they exist in practice) are related to OpenAI specific policies in how they are approaching developing their models in the fine tuning stages, not to inherent limitations in the underlying tech.