3 ms·

> and yet one year after GPT-4 release, no one has caught up to OpenAI You're wrong, Claude 3 Opus beat OpenAI’s GPT-4 in Elo rating just a month ago. They sin

> and yet one year after GPT-4 release, no one has caught up to OpenAI You're wrong, Claude 3 Opus beat OpenAI’s GPT-4 in Elo rating just a month ago. They since lost the top position, but they are at ELO 1246 compared to GPT-4 at 1250 and GPT-4o at 1287. They are bottlenecked at GPT-4 level with small variations. That's the issue. Why have they stopped advancing? The gap between GPT-3.5 and GPT-4 is so much wider than that between GPT-4 and GPT-4o https://medium.com/@simeon.emanuilov/claude-and-gpt-4-top-leaderboard-in-elo-rating-benchmark-of-llms-dfebebae987c https://medium.com/@simeon.emanuilov/claude-and-gpt-4-top-le... The explanation is that we have exhausted the sources of high quality human text. All providers train on essentially the same corpus. Architecture variation doesn't matter. Data is king, and AI used up all the human data. My take on the path forward - it is easier to imitate than to innovate. LLMs up to now are mostly imitative. In order to become innovative they need to learn from things, not just from people. It's a social process, based on evolution of ideas. It's not unlike DNA and language, intelligence is social. It takes a village to raise a child, it takes a whole world to train an AGI. Humans also take 20 years of experience based training before becoming truly innovative at the edge of human knowledge.

aurareturn 2y ago
>The gap between GPT-3.5 and GPT-4 is so much wider than that between GPT-4 and GPT-4o Why is this an argument? GPT4o is obviously a much smaller model than GPT4 and is not meant to replace GPT4. It's meant to replace GPT3.5 as the free version. The fact that OpenAI can make a much smaller model and outperform or perform on par with the original GPT4 is a good sign. OpenAI is still working on the actual GPT4 replacement.
- wokwokwok 2y ago
  > OpenAI is still working on the actual GPT4 replacement. You speculate. ...but, what we can evaluate is what we can see, and what we can see is that there’s now only a marginal difference between what openAI actually has available and what other vendors actually have available. What we havent seen, is a demo of something that is an order of magnitude more capable, aka. GPT5. Not a demo. Not a post. Not tweet. Not a hint of something concrete. Just some vague hand waving that it’s busy baking, take it on faith, it’s gonna be great. …but let’s be really clear here; what they actually delivered is not great, it is less capable. It is smaller. It’s a consumer model for the masses to talk to. So, in that context, you have to take in faith that openAI is going to deliver a GPT5, before anyone else. …but that’s a hunch. That’s a guess. …and they have some very stiff competition; and it’s very very much not clear to me they’re going to the first ones to drop something that is concretely better than gpt4; because gpt-4o was a big disappointment. > The fact that OpenAI can make a much smaller model Everyone is making small models. That is not something I am impressed by.
sashank_1509 2y ago
GPT4o didn’t even need to be better than GPT4 in any benchmarks. Being conversational and running in real time, is such a big difference user experience wise, that the fact that they beat GPT4 is just cherries on top. And it’s almost quasi confirmed that GPT-5 has been trained and that it’s undergoing red teaming and safety checks now and it will be out in the summer, further cementing their 1 year lead.

aurareturn 2y ago

>The gap between GPT-3.5 and GPT-4 is so much wider than that between GPT-4 and GPT-4o Why is this an argument? GPT4o is obviously a much smaller model than GPT4 and is not meant to replace GPT4. It's meant to replace GPT3.5 as the free version. The fact that OpenAI can make a much smaller model and outperform or perform on par with the original GPT4 is a good sign. OpenAI is still working on the actual GPT4 replacement.

wokwokwok 2y ago
> OpenAI is still working on the actual GPT4 replacement. You speculate. ...but, what we can evaluate is what we can see, and what we can see is that there’s now only a marginal difference between what openAI actually has available and what other vendors actually have available. What we havent seen, is a demo of something that is an order of magnitude more capable, aka. GPT5. Not a demo. Not a post. Not tweet. Not a hint of something concrete. Just some vague hand waving that it’s busy baking, take it on faith, it’s gonna be great. …but let’s be really clear here; what they actually delivered is not great, it is less capable. It is smaller. It’s a consumer model for the masses to talk to. So, in that context, you have to take in faith that openAI is going to deliver a GPT5, before anyone else. …but that’s a hunch. That’s a guess. …and they have some very stiff competition; and it’s very very much not clear to me they’re going to the first ones to drop something that is concretely better than gpt4; because gpt-4o was a big disappointment. > The fact that OpenAI can make a much smaller model Everyone is making small models. That is not something I am impressed by.

sashank_1509 2y ago

GPT4o didn’t even need to be better than GPT4 in any benchmarks. Being conversational and running in real time, is such a big difference user experience wise, that the fact that they beat GPT4 is just cherries on top. And it’s almost quasi confirmed that GPT-5 has been trained and that it’s undergoing red teaming and safety checks now and it will be out in the summer, further cementing their 1 year lead.