3 ms·

OpenAI's GPT-5 Pushed Back to Late 2025, but Promises PhD-Level Abilities

10 points by saikatsg 2y ago

Bluestein 2y ago
> PhD-Level Abilities Is not promising that, PhD-Level abilities, basically, promising AGI? (And, the timeframe would allow the hype/expectations to subside ...)
- giardini 2y ago
  Have you worked with a PhD? They can be a difficult fit for private enterprise.
  - Bluestein 2y ago
    Fair enough.-
  - OutOfHere 2y ago
    I have at times seen some of this even in the responses of 3.5 vs 4. But overall, the higher version always wins. The key is to use the right version for the right task.
- dragonwriter 2y ago
  toddler-level abilities would also be AGI.
  - Bluestein 2y ago
    A toddler-level AGI. a "Baby AGI" if you will ...
giardini 2y ago
Thus OpenAI becomes unemployable. See the chart (at bottom of page) labelled "Not So Very Serious Stuff" at https://philip.greenspun.com/careers/ https://philip.greenspun.com/careers/
Strongbad536 2y ago
Tin foil hat theory - they were waiting for training data from their deal with Apple to train GPT-5.
- deleted 2y ago
  [deleted]
tensor 2y ago
Perhaps they should start with "undergrad level abilities." Pretty sure even undergrads can write a paper without made up references.
- OutOfHere 2y ago
  They claim that it jumped from toddler to high-school, so jumping to PhD does not break the pattern. One can argue that undergrad degrees are just a marketing gimmick and were not really meant to exist in a true academic sense.
  - tensor 2y ago
    Unless you look at the actual objective functions for training: primarily, produce human like text, and second, produce answers that "feel good" to a human via reinforcement learning. Neither function optimizes for an understanding of what truth means, generating confidences, reasoning, let alone higher level objectives like "doing PhD level research". Given that none of these things are changing I wouldn't expect any major difference in its ability to provide truthful reasoned responses. It make continue to pick up some of these abilities as a coincidental byproduct of learning to generate text, but it will still be a glorified text generator without these changes. That said maybe they will have some new training objective function, we'll see. Either way these systems are a far cry from human-level when it comes to having actual goals beyond being a toy text generator.
  - dragonwriter 2y ago
    > It jumped from toddler to high-school, No, it didn't. The idea that it did is based on manifestly false assumption that correlations between specific tests and general mental ability that apply on-average to humans apply to LLMs. We can easily observe this is false, but flim-flam salesmen expect us to ignore it in favor of exaggerated hype that obscures rather than assists any meaningful discussion of the utility and capacity of LLMs.
riffraff 2y ago
any idea what the cost for training GPT-5 is going to be? Dario Amodei of Anthropic threw a ballpark number[0] for training future LLMs which is frankly astonishing > The models that are in training now and that will come out at various times later this year or early next year are closer in cost to $1 billion. So that’s already happening. And then I think in 2025 and 2026, we’ll get more towards $5 or $10 billion. but maybe there have been some improvement in efficiency? [0] https://www.nytimes.com/2024/04/12/opinion/ezra-klein-podcast-dario-amodei.html?showTranscript=1 https://www.nytimes.com/2024/04/12/opinion/ezra-klein-podcas...
- dragonwriter 2y ago
  Because more money will deliver more quality even with efficiency gains as long as we don't hit a ceiling on ability to improve with more resources, the cost of new top-end SotA models will scale with hiw much capital is attracted to AI, and efficiency will not reduce top-end SotA model costs. (It may reduce “good enough” model costs for many tasks, but that doesn’t effect the top of the market.)
spicyusername 2y ago
Alternate theory: Open AI has run out of steam, but the second the market discovers that they don't know how to improve their products any further, the AI bubble will pop. Better to just kick the can down the road and pretend something is coming while the money flows in.
- dvfjsdhgfv 2y ago
  While I agree with that to some degree, I believe it's not that they can't make better models, it's just the improvements aren't that spectacular, and OpenAI needs to be spectacular, they can't afford to be boring tech like Gemini. (I tested Claude 3.5 Sonnet on my custom task set recently and it beat ChatGPT-4 in several tiny but important details which means there is definitely a room for improvement for OpenAI)
Workaccount2 2y ago
Just want to say: We have no idea what the size or scope of the spectrum of intelligence is. Everything about intelligence that people discuss uses the human brain as the measuring stick. However, it's unlikely that the human intelligence measuring stick is anything more than a rounding error on the total span of possible intelligence. The same way astronomers ditched the kilometer once they discovered how vast space was. To put it another way, I think it's very foolish to think passing human intelligence (once the right scaling model is found) will be any more difficult than making a 1GB SD card or achieving 100 MIPS. We'll probably blow right past it.
- jasfi 2y ago
  We have standardized tests to measure intelligence. These are always compared when a top new model comes out.
krackers 2y ago
Forget GPT-5, they haven't even released 4o with voice yet. I guess they're competing with google for vaporware.

10 points by saikatsg 2y ago

> PhD-Level Abilities Is not promising that, PhD-Level abilities, basically, promising AGI? (And, the timeframe would allow the hype/expectations to subside ...)

giardini 2y ago
Have you worked with a PhD? They can be a difficult fit for private enterprise.
- Bluestein 2y ago
  Fair enough.-
- OutOfHere 2y ago
  I have at times seen some of this even in the responses of 3.5 vs 4. But overall, the higher version always wins. The key is to use the right version for the right task.
dragonwriter 2y ago
toddler-level abilities would also be AGI.
- Bluestein 2y ago
  A toddler-level AGI. a "Baby AGI" if you will ...

giardini 2y ago

Thus OpenAI becomes unemployable. See the chart (at bottom of page) labelled "Not So Very Serious Stuff" at https://philip.greenspun.com/careers/ https://philip.greenspun.com/careers/

Strongbad536 2y ago

Tin foil hat theory - they were waiting for training data from their deal with Apple to train GPT-5.

deleted 2y ago
[deleted]

tensor 2y ago

Perhaps they should start with "undergrad level abilities." Pretty sure even undergrads can write a paper without made up references.

OutOfHere 2y ago
They claim that it jumped from toddler to high-school, so jumping to PhD does not break the pattern. One can argue that undergrad degrees are just a marketing gimmick and were not really meant to exist in a true academic sense.
- tensor 2y ago
  Unless you look at the actual objective functions for training: primarily, produce human like text, and second, produce answers that "feel good" to a human via reinforcement learning. Neither function optimizes for an understanding of what truth means, generating confidences, reasoning, let alone higher level objectives like "doing PhD level research". Given that none of these things are changing I wouldn't expect any major difference in its ability to provide truthful reasoned responses. It make continue to pick up some of these abilities as a coincidental byproduct of learning to generate text, but it will still be a glorified text generator without these changes. That said maybe they will have some new training objective function, we'll see. Either way these systems are a far cry from human-level when it comes to having actual goals beyond being a toy text generator.
- dragonwriter 2y ago
  > It jumped from toddler to high-school, No, it didn't. The idea that it did is based on manifestly false assumption that correlations between specific tests and general mental ability that apply on-average to humans apply to LLMs. We can easily observe this is false, but flim-flam salesmen expect us to ignore it in favor of exaggerated hype that obscures rather than assists any meaningful discussion of the utility and capacity of LLMs.

riffraff 2y ago

any idea what the cost for training GPT-5 is going to be? Dario Amodei of Anthropic threw a ballpark number[0] for training future LLMs which is frankly astonishing > The models that are in training now and that will come out at various times later this year or early next year are closer in cost to $1 billion. So that’s already happening. And then I think in 2025 and 2026, we’ll get more towards $5 or $10 billion. but maybe there have been some improvement in efficiency? [0] https://www.nytimes.com/2024/04/12/opinion/ezra-klein-podcast-dario-amodei.html?showTranscript=1 https://www.nytimes.com/2024/04/12/opinion/ezra-klein-podcas...

dragonwriter 2y ago
Because more money will deliver more quality even with efficiency gains as long as we don't hit a ceiling on ability to improve with more resources, the cost of new top-end SotA models will scale with hiw much capital is attracted to AI, and efficiency will not reduce top-end SotA model costs. (It may reduce “good enough” model costs for many tasks, but that doesn’t effect the top of the market.)

spicyusername 2y ago

Alternate theory: Open AI has run out of steam, but the second the market discovers that they don't know how to improve their products any further, the AI bubble will pop. Better to just kick the can down the road and pretend something is coming while the money flows in.

dvfjsdhgfv 2y ago
While I agree with that to some degree, I believe it's not that they can't make better models, it's just the improvements aren't that spectacular, and OpenAI needs to be spectacular, they can't afford to be boring tech like Gemini. (I tested Claude 3.5 Sonnet on my custom task set recently and it beat ChatGPT-4 in several tiny but important details which means there is definitely a room for improvement for OpenAI)

Workaccount2 2y ago

Just want to say: We have no idea what the size or scope of the spectrum of intelligence is. Everything about intelligence that people discuss uses the human brain as the measuring stick. However, it's unlikely that the human intelligence measuring stick is anything more than a rounding error on the total span of possible intelligence. The same way astronomers ditched the kilometer once they discovered how vast space was. To put it another way, I think it's very foolish to think passing human intelligence (once the right scaling model is found) will be any more difficult than making a 1GB SD card or achieving 100 MIPS. We'll probably blow right past it.

jasfi 2y ago
We have standardized tests to measure intelligence. These are always compared when a top new model comes out.

krackers 2y ago

Forget GPT-5, they haven't even released 4o with voice yet. I guess they're competing with google for vaporware.