6 ms·

Just give it a year for this bubble/hype to blow over. We have plateaued since gpt-4 and now most of the industry is hype-driven to get investor money. There is

• why_only_15 2y ago
  how is this a plateau since gpt-4? this is significantly better
  • kenjackson 2y ago
    People act as if GPT-4 came out 10 years ago.
  • csomar 2y ago
    First, this model is yet to be released. This is a momentum "announcement". When the O1 was "announced", it was announced as a "breakthrough" but I use Claude/O1 daily and 80% of the time Claude beats it. I also see it as a highly fine-tuned/targeted GPT-4 rather than something that has complex understanding. So we'll find out if this model is real or not by 2-3 months. My guess is that it'll turn out to be another flop like O1. They needed to release something big because they are momentum based and their ability to raise funding is contingent on their AGI claims.
    • XenophileJKO 2y ago
      I thought o1 was a fine-tune of GPT-4o. I don't think o3 is though. Likely using the same techniques on what would have been the "GPT-5" base model.
  • Jensson 2y ago
    > how is this a plateau since gpt-4? this is significantly better Significantly better at what? A benchmark? That isn't necessarily progress. Many report preferring gpt-4 to the newer o1 models with hidden text. Hidden text makes the model more reliable, but more reliable is bad if it is reliably wrong at something since then you can't ask it over and over to find what you want. I don't feel it is significantly smarter, it is more like having the same dumb person spend more thinking than the model getting smarter.
  • peepeepoopoo97 2y ago
    O3 is multiple orders of magnitude more expensive to realize a marginal performance gain. You could hire 50 full time PhDs for the cost of using O3. You're witnessing the blowoff top of the scaling hype bubble.
    • whynotminot 2y ago
      What they’ve proven here is that it can be done. Now they just have to make it cheap. Tell me, what has this industry been good at since its birth? Driving down the cost of compute and making things more efficient. Are you seriously going to assume that won’t happen here?
      • Jensson 2y ago
        > What they’ve proven here is that it can be done. No they haven't, these results do not generalize, as mentioned in the article: "Furthermore, early data points suggest that the upcoming ARC-AGI-2 benchmark will still pose a significant challenge to o3, potentially reducing its score to under 30% even at high compute" Meaning, they haven't solved AGI, and the task itself do not represent programming well, these model do not perform that well on engineering benchmarks.
        • whynotminot 2y ago
          Sure, AGI hasn’t been solved today. But what they’ve done is show that progress isn’t slowing down. In fact, it looks like things are accelerating. So sure, we’ll be splitting hairs for a while about when we reach AGI. But the point is that just yesterday people were still talking about a plateau.
          • peepeepoopoo97 2y ago
            About 10,000 times the cost for twice the performance sure looks like progress is slowing to me.
            • 2y ago
  • crazylogger 2y ago
    Intelligence has not been LLM's major limiting factor since GPT4. The original GPT4 reports in late-2022 & 2023 already established that it's well beyond an average human in professional fields: https://www.microsoft.com/en-us/research/publication/sparks-of-artificial-general-intelligence-early-experiments-with-gpt-4/ https://www.microsoft.com/en-us/research/publication/sparks-.... They failed to outright replaced humans at work not because of lacking intelligence. We may have progressed from a 99%-accurate chatbot to one that's 99.9%-accurate, and you'd have a hard time telling them apart in normal real world (dumb) applications. A paradigm shift is needed from the current chatbot interface to a long-lived stream of consciousness model (e.g. a brain that constantly reads input and produces thoughts at 10ms refresh rate; remembers events for years and keep the context window from exploding; paired with a cerebellum to drive robot motors, at even higher refresh rates.) As long as we're stuck at chatbots, LLM's impact on the real world will be very limited, regardless of how intelligent they become.
• tigershark 2y ago
  Where is the plateau? Chatgtp 4 was ~0% in ARC-AGI. 4o was 5%. This model literally solved it with a score higher than the 85% of the average human. And let’s not forget the unbelievable 25% in frontier math, where all the most brilliant mathematicians in the world cannot solve by themselves a lot of the problems. We are speaking about cutting edge math research problems that are out of reach from practically everyone. You will get a rude awakening if you call this unbelievable advancement a “plateau”.
  • csomar 2y ago
    I don't care about benchmarks. O1 ranks higher than Claude on "benchmarks" but performs worse on particular real life coding situations. I'll judge the model myself by how useful/correct it is for my tasks rather than a hypothetical benchmarks.
    • whynotminot 2y ago
      “Objective benchmarks are useless, let’s argue about which one works better for me personally.”
      • bakugo 2y ago
        Yes, "objective" benchmarks can be gamed, real-life tasks cannot.
      • csomar 2y ago
        Yes. My benchmarks and their benchmarks means AGI. Their benchmarks only means over-fitted.
        • whynotminot 2y ago
          Ok so what if we get different results for our own personal benchmarks/use cases. (See why objective benchmarks exist?)
    • famouswaffles 2y ago
• dyauspitr 2y ago
  Did you read the article at all? We’re definitely not plateauing.