5 ms·

> If OpenAI had GPT-5, they have would shown it. Not convinced. They had GPT-4 for ages before showing it. >And the most important thing about the figure is t

> If OpenAI had GPT-5, they have would shown it. Not convinced. They had GPT-4 for ages before showing it. >And the most important thing about the figure is that 4o is not a lot different from Turbo, which is not hugely different from 4. It's twice as fast and half the cost. >OpenAI has presumably pivoted to new features precisely because they don’t know how produce the kind of capability advance that the “exponential improvement” would have predicted. Or they think that users would rather have a faster GPT-4 than a smarter but slower one.

birracerveza 2y ago
> Or they think that users would rather have a faster GPT-4 than a smarter but slower one. And they are absolutely right. GPT-4 is already much more than enough for 90% of tasks while maintaining a sane dose of human double-checking. Making it faster enables real-time workflows, and better energy efficiency also gives them much more capability to serve more requests/users while lowering costs. And that knowledge likely carries over to GPT-5 or whatever.
- gtirloni 2y ago
  So as the article is suggesting, Gen AI has probably peaked.
  - sebzim4500 2y ago
    I don't understand how you got that from his comment
    - gtirloni 2y ago
      When you start focusing on increasing speed and trimming the rough edges, the technology is maturing. It can't be maturing AND making huge leaps expected of GPT-5.
  - muyuu 2y ago
    even if the tech didn't improve one iota in the next 10 years, the low hanging fruit that remains for the taking in application is just staggering right now
- jgalt212 2y ago
  > GPT-4 is already much more than enough for 90% of tasks while maintaining a sane dose of human double-checking. There's the rub. Does the cost of double checking a 90% solution best the cost of current methods?
  - madamelic 2y ago
    Yes. Absolutely. Who cares if I have to scan through it when it can write 100 lines of rote boring code in 10 seconds when it would take me 5 minutes. I can type fast but GPT can type thousands of words per minute. I can't compete with that.
    - fidotron 2y ago
      Debugging is way harder than getting it right first time though. The fun thing with all this AI stuff is people are beginning to appreciate the nice thing about computers the whole time was their strict mechanical nature.
- somnic 2y ago
  My duckduckgo results are starting to have summaries that do not reflect the content of the associated site and contain plausible falsehoods, courtesy of bing, and the content-farming keyword-spamming AI generated SEO slop goes without saying at this point. It'd be very nice if these models weren't also polluting the resources that people use to try and verify things.
londons_explore 2y ago
> It's twice as fast and half the cost. There are plenty of usecases that just want the smartest possible AI, and could easily afford 1000x the price before paying a human expert. How much would you pay a lawyer to write a letter for you? $200 maybe? Well GPT-4o will do it for $0.0025. Would you pay 1000x more ($2.50) for an AI which will shoot you with fewer legal footguns? - of course you would. Likewise with time, I am totally happy to wait 10 minutes for a better result, rather than having a mediocre result in 10 seconds.
- paulcole 2y ago
  > Likewise with time, I am totally happy to wait 10 minutes for a better result, rather than having a mediocre result in 10 seconds theres a massive gulf separating what people say they want and what people want. As well as a massive gulf separating what they will pay and what they say they’ll pay. while there are plenty of situations where you think you want the smartest possible ai, it’s clear that a pretty smart pretty quick one is good enough for a lot of things.
- sebzim4500 2y ago
  It could be that using prompt engineering/search techniques like CoT or ToT gives a bigger improvement than scaling up the model.
  - waldrews 2y ago
    Exactly. In the long run we want a cheap/fast LLM layer that we can call millions of times in a search/conversation-with-itself pattern. It may be that we'll interact with it the same way, and GPT-6 will be lots of GPT-5's talking to each other transparently to the user, but the path to next-level intelligence needs to get past generating the next token. Since we don't know the real GPT-4 architecture, it may be doing some of this already.
    - sebzim4500 2y ago
      I think we can be pretty confident GPT-4 isn't doing something like this based on its performance characteristics. i) The complexity of the prompt/answer does not effect the time per token. ii) We start getting a response pretty quickly (for short prompts) and then get new tokens at a roughly constant rate I don't think either of these properties would hold if there were a bunch of models that had to coordinate before they could start writing the output. Certainly techniques like ToT violate them.
rohansood15 2y ago
[dead]
kappuchino 2y ago
> It's twice as fast and half the cost. is a great trick with language. Because if you just pay for computing time, half the time equals half the cost. But when you phrase it that way, it sounds for some people like 4 times. So ...
- sebzim4500 2y ago
  For a lot of things you need to pay more for more speed, so that's why I specified both. Otherwise they could probably make it way faster but also much more expensive by doing less batching, speculative decoding, etc.
reissbaker 2y ago
And not only is it twice as fast at half the cost, while benchmarking better... It also is multimodal over audio, including understanding turn-taking and ad hoc interruptions in human conversation and recognizing multiple speakers. That's a lot of extra capabilities — not every advancement is measured in logical reasoning on text. Gary Marcus was an advocate of neuro-symbolic approaches to AI, as opposed to deep learning, and he's been incorrectly predicting that "Deep learning is hitting a wall" since the GPT-2 days. His Substack is near-daily low-content negative articles about LLMs, and on Twitter he posts multiple times a day about it and hasn't stopped for years.

birracerveza 2y ago

> Or they think that users would rather have a faster GPT-4 than a smarter but slower one. And they are absolutely right. GPT-4 is already much more than enough for 90% of tasks while maintaining a sane dose of human double-checking. Making it faster enables real-time workflows, and better energy efficiency also gives them much more capability to serve more requests/users while lowering costs. And that knowledge likely carries over to GPT-5 or whatever.

gtirloni 2y ago
So as the article is suggesting, Gen AI has probably peaked.
- sebzim4500 2y ago
  I don't understand how you got that from his comment
  - gtirloni 2y ago
    When you start focusing on increasing speed and trimming the rough edges, the technology is maturing. It can't be maturing AND making huge leaps expected of GPT-5.
- muyuu 2y ago
  even if the tech didn't improve one iota in the next 10 years, the low hanging fruit that remains for the taking in application is just staggering right now
jgalt212 2y ago
> GPT-4 is already much more than enough for 90% of tasks while maintaining a sane dose of human double-checking. There's the rub. Does the cost of double checking a 90% solution best the cost of current methods?
- madamelic 2y ago
  Yes. Absolutely. Who cares if I have to scan through it when it can write 100 lines of rote boring code in 10 seconds when it would take me 5 minutes. I can type fast but GPT can type thousands of words per minute. I can't compete with that.
  - fidotron 2y ago
    Debugging is way harder than getting it right first time though. The fun thing with all this AI stuff is people are beginning to appreciate the nice thing about computers the whole time was their strict mechanical nature.
somnic 2y ago
My duckduckgo results are starting to have summaries that do not reflect the content of the associated site and contain plausible falsehoods, courtesy of bing, and the content-farming keyword-spamming AI generated SEO slop goes without saying at this point. It'd be very nice if these models weren't also polluting the resources that people use to try and verify things.

londons_explore 2y ago

> It's twice as fast and half the cost. There are plenty of usecases that just want the smartest possible AI, and could easily afford 1000x the price before paying a human expert. How much would you pay a lawyer to write a letter for you? $200 maybe? Well GPT-4o will do it for $0.0025. Would you pay 1000x more ($2.50) for an AI which will shoot you with fewer legal footguns? - of course you would. Likewise with time, I am totally happy to wait 10 minutes for a better result, rather than having a mediocre result in 10 seconds.

paulcole 2y ago
> Likewise with time, I am totally happy to wait 10 minutes for a better result, rather than having a mediocre result in 10 seconds theres a massive gulf separating what people say they want and what people want. As well as a massive gulf separating what they will pay and what they say they’ll pay. while there are plenty of situations where you think you want the smartest possible ai, it’s clear that a pretty smart pretty quick one is good enough for a lot of things.
sebzim4500 2y ago
It could be that using prompt engineering/search techniques like CoT or ToT gives a bigger improvement than scaling up the model.
- waldrews 2y ago
  Exactly. In the long run we want a cheap/fast LLM layer that we can call millions of times in a search/conversation-with-itself pattern. It may be that we'll interact with it the same way, and GPT-6 will be lots of GPT-5's talking to each other transparently to the user, but the path to next-level intelligence needs to get past generating the next token. Since we don't know the real GPT-4 architecture, it may be doing some of this already.
  - sebzim4500 2y ago
    I think we can be pretty confident GPT-4 isn't doing something like this based on its performance characteristics. i) The complexity of the prompt/answer does not effect the time per token. ii) We start getting a response pretty quickly (for short prompts) and then get new tokens at a roughly constant rate I don't think either of these properties would hold if there were a bunch of models that had to coordinate before they could start writing the output. Certainly techniques like ToT violate them.

rohansood15 2y ago

[dead]

kappuchino 2y ago

> It's twice as fast and half the cost. is a great trick with language. Because if you just pay for computing time, half the time equals half the cost. But when you phrase it that way, it sounds for some people like 4 times. So ...

sebzim4500 2y ago
For a lot of things you need to pay more for more speed, so that's why I specified both. Otherwise they could probably make it way faster but also much more expensive by doing less batching, speculative decoding, etc.

reissbaker 2y ago

And not only is it twice as fast at half the cost, while benchmarking better... It also is multimodal over audio, including understanding turn-taking and ad hoc interruptions in human conversation and recognizing multiple speakers. That's a lot of extra capabilities — not every advancement is measured in logical reasoning on text. Gary Marcus was an advocate of neuro-symbolic approaches to AI, as opposed to deep learning, and he's been incorrectly predicting that "Deep learning is hitting a wall" since the GPT-2 days. His Substack is near-daily low-content negative articles about LLMs, and on Twitter he posts multiple times a day about it and hasn't stopped for years.