5 ms·

> Marcus wrote: "GPT-5 hasn't dropped, Sora hasn't shipped, the company had an operating loss of $5 billion last year, there is no obvious moat, Meta is giving

> Marcus wrote: "GPT-5 hasn't dropped, Sora hasn't shipped, the company had an operating loss of $5 billion last year, there is no obvious moat, Meta is giving away similar software for free, many lawsuits pending. > "Yet people are valuing this company at $150 billion dollars. > "Absolutely insane. Investors shouldn't be pouring more money at higher valuations, they should be asking what is going on." I've been saying for a while now that the lack of GPT-5 is a huge red flag for their future. They burned all the hype for GPT-5 on 4o, letting the media call their upcoming model GPT-5 for months before coming out with "4 but cheaper". o1 is impressive but again not a new generation—it's pretty clearly just the result of using 4o's cost savings and throwing tons of extra tokens at the problem behind the scenes in a technique that's easily replicable by the competition. OpenAI has no moat and there has been no serious movement by governments to give them the moat they're so desperately lobbying for. Everything about their actions over the past year—the departures, the weird marketing tactics, the AGI hype, and the restructuring—says to me that Altman is trying to set up an exit before investors realize that the competition has already caught up and he has no plan to regain the lead.

zooq_ai 2y ago
This reminds of all the anti-Tesla "Tesla is dead/bankrupt" in 2017/18 takes from mostly clueless people (Gary Marcus is a certified idiot) who don't know anything about scaling a business, TAM of Knowledge Industry ($30T and growing). Always, happy to take the other side of the bet of popular HN comments. (see META, TSLA)
- lolinder 2y ago
  "The fact that some geniuses were laughed at does not imply that all who are laughed at are geniuses. They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown." You're welcome to whichever side you want to take, but you're not right by the simple virtue of being contrary.
  - cdchn 2y ago
    And not right by virtue of $STOCK_GO_UP unless your entire premise is $STOCK_GO_UP.
  - zooq_ai 2y ago
    Of course, being contrarian and being right is important. Happy to bet on Sam Altman and go short on Gary Marcus and popular HN in this case.
- groby_b 2y ago
  All that proves is that shitty businesses that already have scale take a long while to crumble. (See also: Xitter)
shmatt 2y ago
I would say the strawberry/o1 hype was even worse than the GPT-5 hype There were months worth of articles on how strawberry is considered almost dangerous internally its so smart. I know we only got -mini and -preview but...this doesn't feel like AGI
- lordswork 2y ago
  To be fair, o1 is a major breakthrough in the field. If other AI labs can't crack scaling useful inference compute, OpenAI will maintain a big lead.
  - lolinder 2y ago
    Isn't o1 just applying last year's Tree of Thoughts paper in production? Is there any reason to believe that the other companies will struggle to implement their own? https://github.com/princeton-nlp/tree-of-thought-llm https://github.com/princeton-nlp/tree-of-thought-llm
    - impossiblefork 2y ago
      I don't think it's tree of thoughts at all. I think it's as they say: reinforcement learning applied to cause it to generate a relatively long 'reasoning trace' of some kind from which the answer is obtained through summarisation. I think it's likely a cleverly simplified version of QuietSTaR, with no thought tokens, just one big generation to which the RL is applied. The way I believe it's trained in practice is as follows: they have a bunch of examples, some at the edge of GPT-4s ability to answer, some beyond it, some that GPT-4 can answer if you're lucky with the randomness. Then they give it one of these prompts, generate a fairly long text, maybe 3x the length of the answer, and summarize that to produce the final answer. Then they use REINFORCE to reward the generated texts that increase the probability of the summary being correct.
    - WJW 2y ago
      Not be be nitpicky, but being the first to deploy recent academic research papers to production should count as a breakthrough IMHO.
orionsbelt 2y ago
I don't know if OpenAI has a moat or if they will succeed long term against competitors, and certainly the valuation is high, but pointing to the lack of GPT-5 already is laughable. The speed of improvements - including GPT 4o1 preview and the new voice mode - and the business deals (Apple/Siri, whatever he will use the massive capital he raises for) are astounding. The idea that they have been resting on their laurels and have nothing else to show is just demonstrably untrue to date. Maybe the exec departures are a canary in the coal mine for the future, but I'm willing to at least give them a couple years until the last time they shipped what is magical to me (most recently, yesterday).
- lolinder 2y ago
  > The speed of improvements - including GPT 4o1 preview and the new voice mode - and the business deals (Apple/Siri, whatever he will use the massive capital he raises for) are astounding. They're also rapidly replicable. I don't believe for a moment that Apple is designing their system in a way that doesn't allow switching at a moment's notice, and everything they're doing is copied within 6 months by competitors including LLaMA, which Meta keeps releasing for free. > The idea that they have been resting on their laurels and have nothing else to show is just demonstrably untrue to date. I didn't assert that they were resting on their laurels, I asserted that they have no path forward to the next generation and AGI. If they did have a path they wouldn't keep burning their hype for it on applications of the previous generation that are easy to replicate.
jraines 2y ago
It looks dire but it seems too early for the nearly-always-wrong naysayer crowd to be right & take their victory lap. Maybe not. But I always thought that, especially with their lobbying efforts and desire to project a sense of seriousness (even if unfounded) around those efforts, that it was very unlikely they would release a next-gen model before the US election. I agree that the marketing/messaging, esp on social media, is borderline deranged, swinging between “we are basically The Foundation” and “pweese try my pwoduct i hope u love it (heart hands emoji)”
- lolinder 2y ago
  > it seems too early for the nearly-always-wrong naysayer crowd to be right & take their victory lap. I agree it's too early to call for sure, but just to clarify: The naysayers are nearly always right. We just only remember the times they were wrong.
  - hyggetrold 2y ago
    > The naysayers are nearly always right. We just only remember the times they were wrong. Wait what?
    - lolinder 2y ago
      Nearly every venture of any sort fails. The only times we remember the naysayers are on the rare occasions where they were wrong and the venture succeeded.
      - hyggetrold 2y ago
        Ah okay, that makes sense.
    - artwr 2y ago
      I'll let parent elaborate more on the intent, but the way I interpreted it was : Saying that a startup will fail (i.e. being a naysayer) and being right about that is the most likely outcome due to the current "success" distribution (most businesses/startups fail). Also the most memorable ones are when people were dismissive but ultimately wrong about the viability of the business (like the "dropbox" comment).
alfalfasprout 2y ago
The problem is that we're going through what happened in computer vision all over again. Convnets were getting bigger and bigger. New improvements came about in training efficiency, making medium models better, etc. Until... marginal improvements became more and more marginal. LLMs are going through the same thing now. Better and better every iteration but increasingly we're starting to see a fast approaching wall on what they can really do with the current paradigm.
- bottlepalm 2y ago
  We are literally standing on the precipice of agency. After that, it's over. You see us approaching a wall, I see a cliff.
  - WJW 2y ago
    To stick with the metaphor, it's not super unclear if agency is within grasp or on the other side of an abyss. LLMs are definitely an improvement, but it's not at all clear if they can scale to human-level agency. If they reach that, it's even more unclear if they could ever reach superhuman levels given that all their training data is human-level. And finally, we can see from normal human society that it is hardly ever the smartest humans who achieve the most or rise to the highest levels of power. There is no reason to believe that an AI with agency would an inherent "it's over" scenario
    - philistine 2y ago
      We kill millions of humans with agency every day on this planet. And none of them immediately die if we stop providing them with electrical power ... well I guess a small amount of them do. Anyway, we'll be fine. If the AI come, can they immediately stop us from growing food? How can an AI prevent my orchard from giving apples next year? Sure, it can mess up Facebook, but at this point that'd be a benefit.
      - WJW 2y ago
        I don't want to be a hater but according to the UN about 61 million humans die per year, so that only comes out to ~167k per day rather than millions. Most of those will die from old age too, rather than "being killed". Your main point is true though, even superhuman AI would have a rough time in actual combat. It's just too dependent on electricity and datacenters with known locations to actually have a chance.
sroussey 2y ago
Will there even be a GTP-5? Maybe next up is o2 then o3 and so on.
Janicc 2y ago
What's with his obsession with GPT-5? Altman has conistently been saying that there will be no GPT-5 this year almost since the years beginning. He's acting like OpenAI promised GPT-5 and are unable to release it or something.
ben_w 2y ago
What they were lobbying for until very recently isn't a moat, if anything it's the opposite: "we know anything less capable is fine, focus your attention on us and make sure what we do is safe". What's changed very recently, which is a moat and which they may or may not get, is seven 5 GW data centres — the equivalent of "can we tile half of Rhode Island in PV?"
- lolinder 2y ago
  > we know anything less capable is fine, focus your attention on us and make sure what we do is safe This is a form of regulatory capture. You get a lead against your competition and then persuade governments to make it hard for people to follow you by imposing rules and regulations. Regulatory capture is used in this way to build moats.
  - ben_w 2y ago
    > This is a form of regulatory capture. You get a lead against your competition and then persuade governments to make it hard for people to follow you by imposing rules and regulations. They were not making it hard to follow them. That's the point. They were saying they themselves don't really know what they're doing, while saying open source should be protected from any regulations imposed on them. Hard to go past them, perhaps; but even then, given the scale of compute for training bigger models, the only people who could even try would have an easy time following whatever rules are created.
404mm 2y ago
> o1 is impressive I agree, the capabilities and reasoning are really nice compared to 4o. But my first thought when testing it- how are they going to pay for it.. trivial questions burn 15s and more complex 30-40s of some processing time. How does this scale to millions of users??

zooq_ai 2y ago

This reminds of all the anti-Tesla "Tesla is dead/bankrupt" in 2017/18 takes from mostly clueless people (Gary Marcus is a certified idiot) who don't know anything about scaling a business, TAM of Knowledge Industry ($30T and growing). Always, happy to take the other side of the bet of popular HN comments. (see META, TSLA)

lolinder 2y ago
"The fact that some geniuses were laughed at does not imply that all who are laughed at are geniuses. They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown." You're welcome to whichever side you want to take, but you're not right by the simple virtue of being contrary.
- cdchn 2y ago
  And not right by virtue of $STOCK_GO_UP unless your entire premise is $STOCK_GO_UP.
- zooq_ai 2y ago
  Of course, being contrarian and being right is important. Happy to bet on Sam Altman and go short on Gary Marcus and popular HN in this case.
groby_b 2y ago
All that proves is that shitty businesses that already have scale take a long while to crumble. (See also: Xitter)

shmatt 2y ago

I would say the strawberry/o1 hype was even worse than the GPT-5 hype There were months worth of articles on how strawberry is considered almost dangerous internally its so smart. I know we only got -mini and -preview but...this doesn't feel like AGI

lordswork 2y ago
To be fair, o1 is a major breakthrough in the field. If other AI labs can't crack scaling useful inference compute, OpenAI will maintain a big lead.
- lolinder 2y ago
  Isn't o1 just applying last year's Tree of Thoughts paper in production? Is there any reason to believe that the other companies will struggle to implement their own? https://github.com/princeton-nlp/tree-of-thought-llm https://github.com/princeton-nlp/tree-of-thought-llm
  - impossiblefork 2y ago
    I don't think it's tree of thoughts at all. I think it's as they say: reinforcement learning applied to cause it to generate a relatively long 'reasoning trace' of some kind from which the answer is obtained through summarisation. I think it's likely a cleverly simplified version of QuietSTaR, with no thought tokens, just one big generation to which the RL is applied. The way I believe it's trained in practice is as follows: they have a bunch of examples, some at the edge of GPT-4s ability to answer, some beyond it, some that GPT-4 can answer if you're lucky with the randomness. Then they give it one of these prompts, generate a fairly long text, maybe 3x the length of the answer, and summarize that to produce the final answer. Then they use REINFORCE to reward the generated texts that increase the probability of the summary being correct.
  - WJW 2y ago
    Not be be nitpicky, but being the first to deploy recent academic research papers to production should count as a breakthrough IMHO.

orionsbelt 2y ago

I don't know if OpenAI has a moat or if they will succeed long term against competitors, and certainly the valuation is high, but pointing to the lack of GPT-5 already is laughable. The speed of improvements - including GPT 4o1 preview and the new voice mode - and the business deals (Apple/Siri, whatever he will use the massive capital he raises for) are astounding. The idea that they have been resting on their laurels and have nothing else to show is just demonstrably untrue to date. Maybe the exec departures are a canary in the coal mine for the future, but I'm willing to at least give them a couple years until the last time they shipped what is magical to me (most recently, yesterday).

lolinder 2y ago
> The speed of improvements - including GPT 4o1 preview and the new voice mode - and the business deals (Apple/Siri, whatever he will use the massive capital he raises for) are astounding. They're also rapidly replicable. I don't believe for a moment that Apple is designing their system in a way that doesn't allow switching at a moment's notice, and everything they're doing is copied within 6 months by competitors including LLaMA, which Meta keeps releasing for free. > The idea that they have been resting on their laurels and have nothing else to show is just demonstrably untrue to date. I didn't assert that they were resting on their laurels, I asserted that they have no path forward to the next generation and AGI. If they did have a path they wouldn't keep burning their hype for it on applications of the previous generation that are easy to replicate.

jraines 2y ago

It looks dire but it seems too early for the nearly-always-wrong naysayer crowd to be right & take their victory lap. Maybe not. But I always thought that, especially with their lobbying efforts and desire to project a sense of seriousness (even if unfounded) around those efforts, that it was very unlikely they would release a next-gen model before the US election. I agree that the marketing/messaging, esp on social media, is borderline deranged, swinging between “we are basically The Foundation” and “pweese try my pwoduct i hope u love it (heart hands emoji)”

lolinder 2y ago
> it seems too early for the nearly-always-wrong naysayer crowd to be right & take their victory lap. I agree it's too early to call for sure, but just to clarify: The naysayers are nearly always right. We just only remember the times they were wrong.
- hyggetrold 2y ago
  > The naysayers are nearly always right. We just only remember the times they were wrong. Wait what?
  - lolinder 2y ago
    Nearly every venture of any sort fails. The only times we remember the naysayers are on the rare occasions where they were wrong and the venture succeeded.
    - hyggetrold 2y ago
      Ah okay, that makes sense.
  - artwr 2y ago
    I'll let parent elaborate more on the intent, but the way I interpreted it was : Saying that a startup will fail (i.e. being a naysayer) and being right about that is the most likely outcome due to the current "success" distribution (most businesses/startups fail). Also the most memorable ones are when people were dismissive but ultimately wrong about the viability of the business (like the "dropbox" comment).

alfalfasprout 2y ago

The problem is that we're going through what happened in computer vision all over again. Convnets were getting bigger and bigger. New improvements came about in training efficiency, making medium models better, etc. Until... marginal improvements became more and more marginal. LLMs are going through the same thing now. Better and better every iteration but increasingly we're starting to see a fast approaching wall on what they can really do with the current paradigm.

bottlepalm 2y ago
We are literally standing on the precipice of agency. After that, it's over. You see us approaching a wall, I see a cliff.
- WJW 2y ago
  To stick with the metaphor, it's not super unclear if agency is within grasp or on the other side of an abyss. LLMs are definitely an improvement, but it's not at all clear if they can scale to human-level agency. If they reach that, it's even more unclear if they could ever reach superhuman levels given that all their training data is human-level. And finally, we can see from normal human society that it is hardly ever the smartest humans who achieve the most or rise to the highest levels of power. There is no reason to believe that an AI with agency would an inherent "it's over" scenario
  - philistine 2y ago
    We kill millions of humans with agency every day on this planet. And none of them immediately die if we stop providing them with electrical power ... well I guess a small amount of them do. Anyway, we'll be fine. If the AI come, can they immediately stop us from growing food? How can an AI prevent my orchard from giving apples next year? Sure, it can mess up Facebook, but at this point that'd be a benefit.
    - WJW 2y ago
      I don't want to be a hater but according to the UN about 61 million humans die per year, so that only comes out to ~167k per day rather than millions. Most of those will die from old age too, rather than "being killed". Your main point is true though, even superhuman AI would have a rough time in actual combat. It's just too dependent on electricity and datacenters with known locations to actually have a chance.

sroussey 2y ago

Will there even be a GTP-5? Maybe next up is o2 then o3 and so on.

Janicc 2y ago

What's with his obsession with GPT-5? Altman has conistently been saying that there will be no GPT-5 this year almost since the years beginning. He's acting like OpenAI promised GPT-5 and are unable to release it or something.

ben_w 2y ago

What they were lobbying for until very recently isn't a moat, if anything it's the opposite: "we know anything less capable is fine, focus your attention on us and make sure what we do is safe". What's changed very recently, which is a moat and which they may or may not get, is seven 5 GW data centres — the equivalent of "can we tile half of Rhode Island in PV?"

lolinder 2y ago
> we know anything less capable is fine, focus your attention on us and make sure what we do is safe This is a form of regulatory capture. You get a lead against your competition and then persuade governments to make it hard for people to follow you by imposing rules and regulations. Regulatory capture is used in this way to build moats.
- ben_w 2y ago
  > This is a form of regulatory capture. You get a lead against your competition and then persuade governments to make it hard for people to follow you by imposing rules and regulations. They were not making it hard to follow them. That's the point. They were saying they themselves don't really know what they're doing, while saying open source should be protected from any regulations imposed on them. Hard to go past them, perhaps; but even then, given the scale of compute for training bigger models, the only people who could even try would have an easy time following whatever rules are created.

404mm 2y ago

> o1 is impressive I agree, the capabilities and reasoning are really nice compared to 4o. But my first thought when testing it- how are they going to pay for it.. trivial questions burn 15s and more complex 30-40s of some processing time. How does this scale to millions of users??