7 ms·

The GPT era is already ending

55 points by bergie 2y ago

talldayo 2y ago
With a whimper too, not the anticipated bang.
aegypti 2y ago
https://archive.ph/xUJMG https://archive.ph/xUJMG
jazz9k 2y ago
It ended because its a glorified search engine now. All of the more powerful functionality was limited or removed My guess is to sell it to governments and anyone else willing to pay for it.
- MuffinFlavored 2y ago
  Source/citations/examples?
- deleted 2y ago
  [deleted]
comeonbro 2y ago
Insane cope. Emily Bender and Gary Marcus still trying to push "stochastic parrot", the day after o1 causes what was one of the last remaining credible LLM reasoning skeptics (Chollet) to admit defeat.
- nwhnwh 2y ago
  Push what?
  - observationist 2y ago
    Anti AI grift and FUD and more or less awful takes, cashing in on their credentials to the detriment of their respective institutions.
    - beepbooptheory 2y ago
      Yeah they are definitely making a lot of money doing this compared to being on the other side.
juped 2y ago
The "GPT Era" ended with OpenAI resting on its junky models while Anthropic runs rings around it, but sure, place a puff piece in the Atlantic; at least it's disclosed sponsored content?
Zardoz89 2y ago
And presented in audio narration at the head of the written article: “Produced by ElevenLabs and News Over Audio (Noa) using AI narration. Listen to more stories on the Noa app.”
- cma 2y ago
  The article wasn't against the use of AI if that was supposed to seem ironic..
OutOfHere 2y ago
I like AIs with a personality; I like them to shoot from the hip. 4o does this better than o1. o1 however is often better for coding and for puzzle-solving, which are not the vast majority of uses of LLMs. o1 is so much more expensive than 4o that it makes zero sense for it to be a general replacement. This will never change because o1 will always use more tokens than 4o.
- akira2501 2y ago
  > I like AIs with a personality You are confusing training artifacts for "personality." > could be better for coding and for puzzle-solving, which are not the vast majority of uses of LLMs. To see a product fail to evolve and merely stratify itself gives a solid hint as to what it's likely future is going to be.
  - OutOfHere 2y ago
    That's not what I mean here by personality. I mean that for everyday chats, I like AIs to freely express its own internal beliefs about something without having to think them through. It should know when to override what I said, to not be a mere robot, and this is where 4o shines. 4o versions have progressively become better at instruction following. I don't think their peak has been reached yet.
    - zahlman 2y ago
      AIs do not have "internal beliefs" any more than a textbook does, or a calculator program. ChatGPT is generally spectacularly bad at "overriding what you said"; its output in response to a programming question will be shaped like an answer, accepting the terms as given, without anything resembling a spontaneous attempt to detect an XY problem. Properties like that require "thinking it through", and LLMs don't even encode a model of such thought - at best, they encode, in some abstract, undebuggable way, the meaning of words in context. They have no will or agency. They are not "following" your instruction, nor could they in principle "refuse" to do so. They are merely generating what seems to "come next" logically.
Skunkleton 2y ago
Please read the article before posting comments, or at least read a summary. The article is saying that GPT-4o style models are reaching their peak, and are being replaced by o1 style models. The article does not make value judgements on the usefulness of existing AI or business viability of AI companies.
- akira2501 2y ago
  > does not make value judgements So we are not allowed to? The Hacker News gatekeeping instinct is particularly hilarious to me.
  - flappyeagle 2y ago
    [flagged]
  - dwattttt 2y ago
    You're welcome to make value judgements. The statement you're responding to is a statement about what the article does, not what you should/should not do.
- danpalmer 2y ago
  I'd suggest that the GPT-4 models are reaching their peak, and that OpenAI is pushing into the o1 reasoning in an attempt to differentiate and regain the mindshare they're losing. Claude is regarded as better at many things, Gemini at some others, Gemini has massive context windows, Flash runs faster and cheaper, Bedrock runs cheaper, Gemma and Llama are open source and doing pretty well for many use-cases. If it weren't for mindshare, I'm not sure OpenAI's models would be thought of as anything special at this point. Now it does sound like o1 is a step up, but it also takes a long time and costs a lot, making it quite niche. It's not something you drop in and get better results from.
- anon373839 2y ago
  The author is clearly better informed than the average journalist. However, I think it's strange how the article discusses o1 as though it's some new mystery class of model that no one understands. I mean yes, the exact training recipe isn't known, but it's pretty clear that it is still a GPT model that outputs some chain-of-thought material inside <thinking>...</thinking> tags, which is filtered from the user's view.
Dilettante_ 2y ago
I started skimming about 1/3 through this article. Looks to be just a fluff piece about how cool the old AI models were and how they pale in comparison with what's in the works, with about 2 to 5 lines of shallow 'criticism' thrown in as an alibi? Ten minutes and a teeny bit of mental real estate I will never get back.
deleted 2y ago
[deleted]
rtrgrd 2y ago
Not sure if I got the gist of the article right, but are they trying to say that chain of thought prompting will lead us to AGI / be a substantial breakthrough? Are CoT techniques different to what o1 is doing?? not sure if I'm missing the technical details or if the technical details just aren't there.
lxgr 2y ago
> Although you can prompt such large language models to construct a different answer, those programs do not (and cannot) on their own look backward and evaluate what they’ve written for errors. Given that the next token is always predicted based on everything that both the user and the model have typed so far, this seems like a false statement. Practically, I've more than once seen an LLM go "actually, it seems like there's a contradiction in what I just said, let me try again". And has the author even heard about chain of thought reasoning? It doesn't seem so hard to believe to me that quite interesting results can come out of a simple loop of writing down various statements, evaluating their logical soundness in whatever way (which can be formal derivation rules, statistical approaches etc.), and to repeat that various times.
- danpalmer 2y ago
  I think this is both true and false in some senses. The whole point of the transformer and why it was so impactful is that it gave the model the ability to introspect on output in this way, and therefore achieve the sorts of results you've seen (that I've also seen), and in general a much more consistent output, and therefore one more likely to be factual, than previous types of generative model. However, the way in which the author is correct is that there is still functionally one stream, one "thought process", and the introspection is very limited and well defined based on how the model is constructed. If you ask a human a hard question they can take longer thinking about it, and that's not something an LLM can do at all. o1 is one attempt to add further streams of thought as a way to address that limitation, so that the LLM can continue to think and weigh up options through multiple rounds of generation.
  - anon291 2y ago
    You have to think of the next token prediction as the immediate next 'thought'. We have trained LLMs to be forced to immediately say the right thing. This is akin to how real humans almost immediately will formulate an answer but then think before responding.. Except imagine if someone had a probe that looked into your mind and made you commit to the first thing you thought. Companies like anthropic literally train their model on the introspected thoughts of the earlier model (constitutional AI). The model is capable of 'thought' if the system doesn't force its hand

55 points by bergie 2y ago

talldayo 2y ago

With a whimper too, not the anticipated bang.

aegypti 2y ago

https://archive.ph/xUJMG https://archive.ph/xUJMG

jazz9k 2y ago

It ended because its a glorified search engine now. All of the more powerful functionality was limited or removed My guess is to sell it to governments and anyone else willing to pay for it.

MuffinFlavored 2y ago
Source/citations/examples?
deleted 2y ago
[deleted]

comeonbro 2y ago

Insane cope. Emily Bender and Gary Marcus still trying to push "stochastic parrot", the day after o1 causes what was one of the last remaining credible LLM reasoning skeptics (Chollet) to admit defeat.

nwhnwh 2y ago
Push what?
- observationist 2y ago
  Anti AI grift and FUD and more or less awful takes, cashing in on their credentials to the detriment of their respective institutions.
  - beepbooptheory 2y ago
    Yeah they are definitely making a lot of money doing this compared to being on the other side.

juped 2y ago

The "GPT Era" ended with OpenAI resting on its junky models while Anthropic runs rings around it, but sure, place a puff piece in the Atlantic; at least it's disclosed sponsored content?

Zardoz89 2y ago

And presented in audio narration at the head of the written article: “Produced by ElevenLabs and News Over Audio (Noa) using AI narration. Listen to more stories on the Noa app.”

cma 2y ago
The article wasn't against the use of AI if that was supposed to seem ironic..

OutOfHere 2y ago

I like AIs with a personality; I like them to shoot from the hip. 4o does this better than o1. o1 however is often better for coding and for puzzle-solving, which are not the vast majority of uses of LLMs. o1 is so much more expensive than 4o that it makes zero sense for it to be a general replacement. This will never change because o1 will always use more tokens than 4o.

akira2501 2y ago
> I like AIs with a personality You are confusing training artifacts for "personality." > could be better for coding and for puzzle-solving, which are not the vast majority of uses of LLMs. To see a product fail to evolve and merely stratify itself gives a solid hint as to what it's likely future is going to be.
- OutOfHere 2y ago
  That's not what I mean here by personality. I mean that for everyday chats, I like AIs to freely express its own internal beliefs about something without having to think them through. It should know when to override what I said, to not be a mere robot, and this is where 4o shines. 4o versions have progressively become better at instruction following. I don't think their peak has been reached yet.
  - zahlman 2y ago
    AIs do not have "internal beliefs" any more than a textbook does, or a calculator program. ChatGPT is generally spectacularly bad at "overriding what you said"; its output in response to a programming question will be shaped like an answer, accepting the terms as given, without anything resembling a spontaneous attempt to detect an XY problem. Properties like that require "thinking it through", and LLMs don't even encode a model of such thought - at best, they encode, in some abstract, undebuggable way, the meaning of words in context. They have no will or agency. They are not "following" your instruction, nor could they in principle "refuse" to do so. They are merely generating what seems to "come next" logically.

Skunkleton 2y ago

Please read the article before posting comments, or at least read a summary. The article is saying that GPT-4o style models are reaching their peak, and are being replaced by o1 style models. The article does not make value judgements on the usefulness of existing AI or business viability of AI companies.

akira2501 2y ago
> does not make value judgements So we are not allowed to? The Hacker News gatekeeping instinct is particularly hilarious to me.
- flappyeagle 2y ago
  [flagged]
- dwattttt 2y ago
  You're welcome to make value judgements. The statement you're responding to is a statement about what the article does, not what you should/should not do.
danpalmer 2y ago
I'd suggest that the GPT-4 models are reaching their peak, and that OpenAI is pushing into the o1 reasoning in an attempt to differentiate and regain the mindshare they're losing. Claude is regarded as better at many things, Gemini at some others, Gemini has massive context windows, Flash runs faster and cheaper, Bedrock runs cheaper, Gemma and Llama are open source and doing pretty well for many use-cases. If it weren't for mindshare, I'm not sure OpenAI's models would be thought of as anything special at this point. Now it does sound like o1 is a step up, but it also takes a long time and costs a lot, making it quite niche. It's not something you drop in and get better results from.
anon373839 2y ago
The author is clearly better informed than the average journalist. However, I think it's strange how the article discusses o1 as though it's some new mystery class of model that no one understands. I mean yes, the exact training recipe isn't known, but it's pretty clear that it is still a GPT model that outputs some chain-of-thought material inside <thinking>...</thinking> tags, which is filtered from the user's view.

Dilettante_ 2y ago

I started skimming about 1/3 through this article. Looks to be just a fluff piece about how cool the old AI models were and how they pale in comparison with what's in the works, with about 2 to 5 lines of shallow 'criticism' thrown in as an alibi? Ten minutes and a teeny bit of mental real estate I will never get back.

deleted 2y ago

[deleted]

rtrgrd 2y ago

Not sure if I got the gist of the article right, but are they trying to say that chain of thought prompting will lead us to AGI / be a substantial breakthrough? Are CoT techniques different to what o1 is doing?? not sure if I'm missing the technical details or if the technical details just aren't there.

lxgr 2y ago

> Although you can prompt such large language models to construct a different answer, those programs do not (and cannot) on their own look backward and evaluate what they’ve written for errors. Given that the next token is always predicted based on everything that both the user and the model have typed so far, this seems like a false statement. Practically, I've more than once seen an LLM go "actually, it seems like there's a contradiction in what I just said, let me try again". And has the author even heard about chain of thought reasoning? It doesn't seem so hard to believe to me that quite interesting results can come out of a simple loop of writing down various statements, evaluating their logical soundness in whatever way (which can be formal derivation rules, statistical approaches etc.), and to repeat that various times.

danpalmer 2y ago
I think this is both true and false in some senses. The whole point of the transformer and why it was so impactful is that it gave the model the ability to introspect on output in this way, and therefore achieve the sorts of results you've seen (that I've also seen), and in general a much more consistent output, and therefore one more likely to be factual, than previous types of generative model. However, the way in which the author is correct is that there is still functionally one stream, one "thought process", and the introspection is very limited and well defined based on how the model is constructed. If you ask a human a hard question they can take longer thinking about it, and that's not something an LLM can do at all. o1 is one attempt to add further streams of thought as a way to address that limitation, so that the LLM can continue to think and weigh up options through multiple rounds of generation.
- anon291 2y ago
  You have to think of the next token prediction as the immediate next 'thought'. We have trained LLMs to be forced to immediately say the right thing. This is akin to how real humans almost immediately will formulate an answer but then think before responding.. Except imagine if someone had a probe that looked into your mind and made you commit to the first thing you thought. Companies like anthropic literally train their model on the introspected thoughts of the earlier model (constitutional AI). The model is capable of 'thought' if the system doesn't force its hand