4 ms·

GPT 4.5 also has a knowledge cutoff date of 10-2023. https://www.reddit.com/r/singularity/comments/1izpb8t/gpt45_knowledge_cutoff_is_still_october_2023/ https:

GPT 4.5 also has a knowledge cutoff date of 10-2023. https://www.reddit.com/r/singularity/comments/1izpb8t/gpt45_knowledge_cutoff_is_still_october_2023/ https://www.reddit.com/r/singularity/comments/1izpb8t/gpt45_... I'm guessing that this model was finished pre-training at least a year ago (it's been 2 years since GPT 4.0 was released) and they just didn't see the hoped-for performance gains to think it warranted releasing at the time, and so put all their effort into the Q-star/strawberry = eventual O1 reasoning effort instead. It seems that OpenAI's reasoning model lead isn't perhaps what they thought it was, and the recent slew of strong non-reasoning models (Gemini 2.0 Flash, Grok 3, Sonnet 3.7) made them feel the need to release something themselves for appearances sake, so they dusted off this model, perhaps did a bit of post-training on it for EQ, and here we are. The price is a bit of a mystery - perhaps just a reflection of an older model without all the latest efficiency tricks to make it cheaper. Maybe it's dense rather than MoE - who knows.

sigmoid10 2y ago
Rumors said that GPT4.5 is an order of magnitude larger. Around 12 trillion parameters total (compared to GPT4's 1.2 trillion). It's almost certainly MoE as well, just a scaled up version. That would explain the cost. OpenAI also said that this is what they originally developed as "Omni" - the model supposed to succeed GPT4 but which fell behind expectations. So they renamed it 4.5 and shoehorned it in to remain in the news among all those competitor releases.
- ljlolel 2y ago
  the gpt-4o ("omni") is probably a distilled 4.5; hence why not much quality difference
  - sigmoid10 2y ago
    4o has been out since May last year, while omni (now rechristened as 4.5) only finished training in October/November.
    - cubefox 2y ago
      4.5 was called Orion, not Omni.
- Leary 2y ago
  How does this compare with Grok 3's parameter count? I know Grok 3 was trained on a larger cluster (100k-200k) but GPT 4.5 used distributed training.
- glenstein 2y ago
  This is all excellent detail. Wondering if there's any good suggestions for further reading on the inside baseball of what happened with GPT 4.5?
  - qeternity 2y ago
    Well, it's not...it gets most details wrong.
    - glenstein 2y ago
      Can you elaborate?
      - qeternity 2y ago
        GPT-4 was rumored to be 1.8T params...not 1.2 And the successor model was called "Orion", not "Omni".
        glenstein 2y ago
        Appreciate the corrections, but I'm still a bit puzzled. Are they wrong about 4.5 having 12 trillion parameters, it originally intending to be Orion (not omni), or an expected successor to GPT 4? And do you have any related reading that speaks to any of this?
        az226 2y ago
        GPT-4 was 1.3T. 221B active. 2 experts active. 16 experts total.
        qeternity 2y ago
        That sounds correct except for that total parameter count. 110B per expert at 16 experts puts you just shy of 1.8T. Are you suggesting there are ca. 30B shared params between experts?
- qeternity 2y ago
  GPT-4 was rumored to be 1.8T params...not 1.2 And the successor model was called "Orion", not "Omni".
- zaptrem 2y ago
  You're thinking of "Orion" not "Omni" (GPT 4o stands for "Omni" since it's natively multimodal with image and audio input/output tokens)
Chance-Device 2y ago
Releasing it was probably a mistake. In context what the model is could have been understood, but they haven’t really presented that context. Also it would be lost on a general audience. The general public will naturally expect it to be the next big thing. Wasn’t that the point of releasing it? To seem like progress is being made? To try to make that point with a model that doesn’t deliver is a misstep. If I were Sam Altman, I’d be pulling this back before it goes on general release, saying something like it was experimental and after user feedback the costs weren’t worth it and they’re working on something else as a replacement. Then o3 or whatever they actually are working on instead can be the “replacement” even if it’s much later.
- deleted 2y ago
  [deleted]
- datadrivenangel 2y ago
  or just say it was too good and thus too dangerous to release...
simonw 2y ago
I don't think th October 2023 training cut-off means the model finished pre-training a year ago. All of OpenAI's models share that same cut-off date. One theory is that they're worried about the increasing tide of LLM-generated slop that's been posted online since that date. I don't know if I buy that or not - other model providers (such as Anthropic, Gemini) don't seem worried about that.
glenstein 2y ago
>The price is a bit of a mystery I think it at least is somewhat analogous to what happened with pricing on previous models. GPT 4, despite being less capable than 4o, is an order of magnitude more expensive, and comparably expensive to o1. It seems like once the model is out, the price is the price, and the performance gains emerge but they emerge attached to new minified variations of previous models.
bilater 2y ago
I sort of believed this but also 4.5 coming out last year would absolutely have been a big deal compared to what was out there at the time? I just dont understand why they would not launch it then.
- deleted 2y ago
  [deleted]
numba888 2y ago
> slew of strong non-reasoning models (Gemini 2.0 Flash, Grok 3, Sonnet 3.7) Sonnet 3.7 is actually reasoning model.
- LaurensBER 2y ago
  It's my understanding that reasoning in Sonnet 3.7 is optional and configurable. I might be wrong but I couldn't find a source that indicates that the "base" model also implements reasoning.
  - amluto 2y ago
    From limited experimentation: Sonnet 3.7 has “extended thinking” as an option, although the UI, at least in the app, leaves something to be desired. It also has a beta feature called “Analysis” that seems to work by having the model output JavaScript code as part of its response that is then run and feeds back into the answer. Both of these abilities are visible — users can see the chain of thought and the analysis code. It seems, based again on limited experimentation doing sort-of-real work, that analysis works quite well and extended thinking is so-so. Whereas DeepSeek R1 seems to be willing and perhaps even encouraged to second-guess itself (maybe this is a superpower of the “wait” token”), Sonnet 3.7 doesn’t seem to second-guess itself as much as it should. It will happily extended-think, generate a wrong answer, and then give a better answer after being asked a question that it really should have thought of itself. (I’m not complaining. I’ve been a happy user of 3.7 for a whole day! But I think there’s plenty of room for improvement.)
- wegfawefgawefg 2y ago
  so is grok3
unsupp0rted 2y ago
> GPT 4.5 also has a knowledge cutoff date of 10-2023. I've heard the theory that's intentional, because after roughly that point the internet training data became full of AI slop

sigmoid10 2y ago

Rumors said that GPT4.5 is an order of magnitude larger. Around 12 trillion parameters total (compared to GPT4's 1.2 trillion). It's almost certainly MoE as well, just a scaled up version. That would explain the cost. OpenAI also said that this is what they originally developed as "Omni" - the model supposed to succeed GPT4 but which fell behind expectations. So they renamed it 4.5 and shoehorned it in to remain in the news among all those competitor releases.

ljlolel 2y ago
the gpt-4o ("omni") is probably a distilled 4.5; hence why not much quality difference
- sigmoid10 2y ago
  4o has been out since May last year, while omni (now rechristened as 4.5) only finished training in October/November.
  - cubefox 2y ago
    4.5 was called Orion, not Omni.
Leary 2y ago
How does this compare with Grok 3's parameter count? I know Grok 3 was trained on a larger cluster (100k-200k) but GPT 4.5 used distributed training.
glenstein 2y ago
This is all excellent detail. Wondering if there's any good suggestions for further reading on the inside baseball of what happened with GPT 4.5?
- qeternity 2y ago
  Well, it's not...it gets most details wrong.
  - glenstein 2y ago
    Can you elaborate?
    - qeternity 2y ago
      GPT-4 was rumored to be 1.8T params...not 1.2 And the successor model was called "Orion", not "Omni".
      - glenstein 2y ago
        Appreciate the corrections, but I'm still a bit puzzled. Are they wrong about 4.5 having 12 trillion parameters, it originally intending to be Orion (not omni), or an expected successor to GPT 4? And do you have any related reading that speaks to any of this?
      - az226 2y ago
        GPT-4 was 1.3T. 221B active. 2 experts active. 16 experts total.
        qeternity 2y ago
        That sounds correct except for that total parameter count. 110B per expert at 16 experts puts you just shy of 1.8T. Are you suggesting there are ca. 30B shared params between experts?
qeternity 2y ago
GPT-4 was rumored to be 1.8T params...not 1.2 And the successor model was called "Orion", not "Omni".
zaptrem 2y ago
You're thinking of "Orion" not "Omni" (GPT 4o stands for "Omni" since it's natively multimodal with image and audio input/output tokens)

Chance-Device 2y ago

Releasing it was probably a mistake. In context what the model is could have been understood, but they haven’t really presented that context. Also it would be lost on a general audience. The general public will naturally expect it to be the next big thing. Wasn’t that the point of releasing it? To seem like progress is being made? To try to make that point with a model that doesn’t deliver is a misstep. If I were Sam Altman, I’d be pulling this back before it goes on general release, saying something like it was experimental and after user feedback the costs weren’t worth it and they’re working on something else as a replacement. Then o3 or whatever they actually are working on instead can be the “replacement” even if it’s much later.

deleted 2y ago
[deleted]
datadrivenangel 2y ago
or just say it was too good and thus too dangerous to release...

simonw 2y ago

I don't think th October 2023 training cut-off means the model finished pre-training a year ago. All of OpenAI's models share that same cut-off date. One theory is that they're worried about the increasing tide of LLM-generated slop that's been posted online since that date. I don't know if I buy that or not - other model providers (such as Anthropic, Gemini) don't seem worried about that.

glenstein 2y ago

>The price is a bit of a mystery I think it at least is somewhat analogous to what happened with pricing on previous models. GPT 4, despite being less capable than 4o, is an order of magnitude more expensive, and comparably expensive to o1. It seems like once the model is out, the price is the price, and the performance gains emerge but they emerge attached to new minified variations of previous models.

bilater 2y ago

I sort of believed this but also 4.5 coming out last year would absolutely have been a big deal compared to what was out there at the time? I just dont understand why they would not launch it then.

deleted 2y ago
[deleted]

numba888 2y ago

> slew of strong non-reasoning models (Gemini 2.0 Flash, Grok 3, Sonnet 3.7) Sonnet 3.7 is actually reasoning model.

LaurensBER 2y ago
It's my understanding that reasoning in Sonnet 3.7 is optional and configurable. I might be wrong but I couldn't find a source that indicates that the "base" model also implements reasoning.
- amluto 2y ago
  From limited experimentation: Sonnet 3.7 has “extended thinking” as an option, although the UI, at least in the app, leaves something to be desired. It also has a beta feature called “Analysis” that seems to work by having the model output JavaScript code as part of its response that is then run and feeds back into the answer. Both of these abilities are visible — users can see the chain of thought and the analysis code. It seems, based again on limited experimentation doing sort-of-real work, that analysis works quite well and extended thinking is so-so. Whereas DeepSeek R1 seems to be willing and perhaps even encouraged to second-guess itself (maybe this is a superpower of the “wait” token”), Sonnet 3.7 doesn’t seem to second-guess itself as much as it should. It will happily extended-think, generate a wrong answer, and then give a better answer after being asked a question that it really should have thought of itself. (I’m not complaining. I’ve been a happy user of 3.7 for a whole day! But I think there’s plenty of room for improvement.)
wegfawefgawefg 2y ago
so is grok3

unsupp0rted 2y ago

> GPT 4.5 also has a knowledge cutoff date of 10-2023. I've heard the theory that's intentional, because after roughly that point the internet training data became full of AI slop