4 ms·

GPT-5 is a product that represents the needs of its users. They have 700M weekly active users, the vast majority of which just want a better google, therapist,

by 827a 1y ago

GPT-5 is a product that represents the needs of its users. They have 700M weekly active users, the vast majority of which just want a better google, therapist, life coach, or programming mate; not some superintelligent being that can solve the Riemann hypothesis. The reason why true singularity-level ASI will never happen isn't because the technology cannot support it (though, to be clear: It can't). Its because no one actually wants it. The capital markets will take care of the rest.

mlyle 1y ago
> The reason why true singularity-level ASI will never happen isn't because the technology cannot support it (though, to be clear: It can't). Its because no one actually wants it. Plenty of people want something capable of doing tasks well beyond what GPT-5 can and that are equally capable to a proficient human. If you can do that cheaper or faster or more available than said skilled human, there is definitely a market for it.
- esalman 1y ago
  Nobody will pay them 1.5m bonus for developing ASI though. As much as we hate on meta, open models are the answer.
- 827a 1y ago
  The issue is, what you're describing actually doesn't require intelligence, beyond a point. A mathematician who can solve the Reimann hypothesis isn't likely to be all that good of an L8 AWS Engineer; but what AWS would pay a lot of money for is a digital L8 AWS Engineer, not a digital Terrance Tao. The deep reflection that AI frontier labs have to do, soon, is: If its not intellect, what are we missing?
  - Avicebron 1y ago
    embodiment?
  - klipt 1y ago
    I'm sure plenty of cancer patients would pay good money for a digital genius doctor who can custom design a drug to target their cancer.
    - 827a 1y ago
      Again, this doesn't require intelligence, beyond a point. Breakthrough medical therapies aren't invented by 180 IQ Einsteinian hyper-intellectuals. My point in being pedantic about this is to just point out that an extreme amount of value could be generated by these systems if they only attained the capability to do the things 110 IQ humans already do today. Instead of optimizing toward that goal, the frontier labs seem obsessed with optimizing toward some other weird goal of hyper-intelligence that has very unclear marketability. When pressed on this, leaders in these companies will say "its transferable"; that hyper-intellectuality will suddenly unlock realistic 110 IQ use-cases; but we're increasingly seeing, quite clearly to me, that this isn't the case. And there's very good first principal reasons why you should believe that it won't be.
      - Gud 1y ago
        No, because Einstein-level intelligence is rare. But 180 IQ intelligence that doesn’t sleep doesn’t stop with instant access to the worlds knowledge will absolutely be able to revolutionise the sciences.
        mlyle 1y ago
        So, I'm not sure 180 IQ intelligence can. But something that has equivalence to high human intelligence (say 140IQ, generalized to a wide variety of domains) and can engage in long-term directed, purposeful, and autonomous behavior will (not all humans of high intelligence cross this bar, but the ones that do tend to make an impact).
      - mlyle 1y ago
        I would argue that AI is able to do a lot of what a 110 IQ human does today already-- at least the 30% of the population that is of average intelligence and has a below average appetite for growth and challenge. And sure, it's not perfectly transferable. It's going to be a long time before it can fix my plumbing, but there's a fair number of tasks that it can do now that are useful. > toward some other weird goal of hyper-intelligence I think the near term goal is something that can do most human tasks of the scope of tens of minutes of work at a computer, without task selection or self-direction. In some ways this looks superhuman, doing this stuff at high speed and 24/7. But it's superhuman like a mechanical loom is superhuman. Even better would be something you could delegate higher level goals, including tasks of larger scope and duration, but that's slower coming.
  - danaris 1y ago
    > If its not intellect, what are we missing? ...Why would it not be intellect? What we're missing is anything that actually resembles thought at anything more than the very surface level. LLMs are not intelligent. They sound intelligent because they're being trained very well at predicting what tokens to produce to sound that way, but they have no intellect, no understanding, no world model, no consciousness—no intelligence.
    - seba_dos1 1y ago
      I love how in the world of LLMs "reasoning" means having the model write out things like "I'm collecting a list of resources" just so it can then better cosplay someone who did that. I'm eagerly waiting for the next big breakthrough when someone trains a model that switches between "thinking" and "outputting" modes mid-answer :)
      - mlyle 1y ago
        > I'm eagerly waiting for the next big breakthrough when someone trains a model that switches between "thinking" and "outputting" modes mid-answer :) ??? That was last year's breakthrough.
        seba_dos1 1y ago
        See??? What a breakneck pace! :D
        mlyle 1y ago
        So, I guess it was just a low effort comment, and I shouldn't take you seriously? I thought you were making an actual effort to converse. Generating tokens to represent steps isn't just "cosplaying" --- it's representing more state than can fit into a single step of the model. Ditto for when I talk to myself when figuring something out, either out loud or just mentally.
        seba_dos1 1y ago
techpineapple 1y ago
> just want a better google, therapist, life coach, or programming mate; not some superintelligent being that can solve the Riemann hypothesis Except it seems like it's a worse google, therapist, life coach and programming mate, with the personality of someone who spends all their time trying to solve the Riemann hypothesis.
- mlyle 1y ago
  To me, GPT-5 feels slightly better. And rumor is the cost for OpenAI to provide it is much less.
- 827a 1y ago
  That's probably true for the time being, though I'd consider that a simple miscalculation on their part than the grand expression of market dynamics that I asserted. They're course-correcting on this and are planning to add more warmth back to its responses.
- georgemcbay 1y ago
  Any of the LLMs (including google's own) is a better "google" than traditional google search. Not really for any technical reason as much as it was that google was perfectly willing to cede the search war to garbage SEO sites as long as they were driving eyeballs to adsense. But the rest I agree with.
- piva00 1y ago
  I wouldn't say a "worse Google", and I'm also tired of the hallucinations and people copy-pasting AI-generated bullshit as if it was correct. It's just a different tool than Google, and quite complimentary in many ways: instead of doing many keywords searches to collect information, aggregate it in a mental model, and derive a conclusion, it's easier to ask it to answer the aggregation I need and back-verify if it makes sense. To me it has helped a lot to not have to know the right incantations of keywords for the information I'm trying to find right out of the bat, and from the LLM answer I can backtrace what searches to do to confirm the validity of it. It's a small twist but it has definitely showed me some value in these tools for more general information lookup. The thing is: I still want and need to have a search engine, if for some reason search engines cease to exist and LLMs take their place I will not be able to trust anything...
aurareturn 1y ago
I agree with your first part. There are a ton of people who want to use LLMs to solve the Riemann hypothesis but those people will need a different model with vastly more compute than just regular old ChatGPT. ASI isn't what people want or not. It's an AGI that is able to self improve at a rapid rate. We haven't built something that can self improve continuously yet. It's not related to whether people want it or not. Personally, I do want ASI.
- 827a 1y ago
  I simply disagree on that specific definition of ASI. Sure, what we're both describing is a system that is able to self-improve at a rapid rate. But, what does "improve" mean? Is an AI which is able to rapidly self-improve only along the intelligence vertical of making the best sourdough bread actually ASI? > We haven't built something that can self improve continuously yet. Actually, we have. Its called AlphaGo; its capable of rapid self-improvement on the intelligence vertical of playing Go. Is that ASI? If your answer to that is No, then you've accepted the base case of a more general idea: That the vertical of self-improvement does matter in the human designation of ASI, which means the objective and reward functions defining "ASI" are at some level human designated. In other words, it is deeply related to what humans want.

by 827a 1y ago

mlyle 1y ago

> The reason why true singularity-level ASI will never happen isn't because the technology cannot support it (though, to be clear: It can't). Its because no one actually wants it. Plenty of people want something capable of doing tasks well beyond what GPT-5 can and that are equally capable to a proficient human. If you can do that cheaper or faster or more available than said skilled human, there is definitely a market for it.

esalman 1y ago
Nobody will pay them 1.5m bonus for developing ASI though. As much as we hate on meta, open models are the answer.
827a 1y ago
The issue is, what you're describing actually doesn't require intelligence, beyond a point. A mathematician who can solve the Reimann hypothesis isn't likely to be all that good of an L8 AWS Engineer; but what AWS would pay a lot of money for is a digital L8 AWS Engineer, not a digital Terrance Tao. The deep reflection that AI frontier labs have to do, soon, is: If its not intellect, what are we missing?
- Avicebron 1y ago
  embodiment?
- klipt 1y ago
  I'm sure plenty of cancer patients would pay good money for a digital genius doctor who can custom design a drug to target their cancer.
  - 827a 1y ago
    Again, this doesn't require intelligence, beyond a point. Breakthrough medical therapies aren't invented by 180 IQ Einsteinian hyper-intellectuals. My point in being pedantic about this is to just point out that an extreme amount of value could be generated by these systems if they only attained the capability to do the things 110 IQ humans already do today. Instead of optimizing toward that goal, the frontier labs seem obsessed with optimizing toward some other weird goal of hyper-intelligence that has very unclear marketability. When pressed on this, leaders in these companies will say "its transferable"; that hyper-intellectuality will suddenly unlock realistic 110 IQ use-cases; but we're increasingly seeing, quite clearly to me, that this isn't the case. And there's very good first principal reasons why you should believe that it won't be.
    - Gud 1y ago
      No, because Einstein-level intelligence is rare. But 180 IQ intelligence that doesn’t sleep doesn’t stop with instant access to the worlds knowledge will absolutely be able to revolutionise the sciences.
      - mlyle 1y ago
        So, I'm not sure 180 IQ intelligence can. But something that has equivalence to high human intelligence (say 140IQ, generalized to a wide variety of domains) and can engage in long-term directed, purposeful, and autonomous behavior will (not all humans of high intelligence cross this bar, but the ones that do tend to make an impact).
    - mlyle 1y ago
      I would argue that AI is able to do a lot of what a 110 IQ human does today already-- at least the 30% of the population that is of average intelligence and has a below average appetite for growth and challenge. And sure, it's not perfectly transferable. It's going to be a long time before it can fix my plumbing, but there's a fair number of tasks that it can do now that are useful. > toward some other weird goal of hyper-intelligence I think the near term goal is something that can do most human tasks of the scope of tens of minutes of work at a computer, without task selection or self-direction. In some ways this looks superhuman, doing this stuff at high speed and 24/7. But it's superhuman like a mechanical loom is superhuman. Even better would be something you could delegate higher level goals, including tasks of larger scope and duration, but that's slower coming.
- danaris 1y ago
  > If its not intellect, what are we missing? ...Why would it not be intellect? What we're missing is anything that actually resembles thought at anything more than the very surface level. LLMs are not intelligent. They sound intelligent because they're being trained very well at predicting what tokens to produce to sound that way, but they have no intellect, no understanding, no world model, no consciousness—no intelligence.
  - seba_dos1 1y ago
    I love how in the world of LLMs "reasoning" means having the model write out things like "I'm collecting a list of resources" just so it can then better cosplay someone who did that. I'm eagerly waiting for the next big breakthrough when someone trains a model that switches between "thinking" and "outputting" modes mid-answer :)
    - mlyle 1y ago
      > I'm eagerly waiting for the next big breakthrough when someone trains a model that switches between "thinking" and "outputting" modes mid-answer :) ??? That was last year's breakthrough.
      - seba_dos1 1y ago
        See??? What a breakneck pace! :D
        mlyle 1y ago
        So, I guess it was just a low effort comment, and I shouldn't take you seriously? I thought you were making an actual effort to converse. Generating tokens to represent steps isn't just "cosplaying" --- it's representing more state than can fit into a single step of the model. Ditto for when I talk to myself when figuring something out, either out loud or just mentally.
        seba_dos1 1y ago

techpineapple 1y ago

> just want a better google, therapist, life coach, or programming mate; not some superintelligent being that can solve the Riemann hypothesis Except it seems like it's a worse google, therapist, life coach and programming mate, with the personality of someone who spends all their time trying to solve the Riemann hypothesis.

mlyle 1y ago
To me, GPT-5 feels slightly better. And rumor is the cost for OpenAI to provide it is much less.
827a 1y ago
That's probably true for the time being, though I'd consider that a simple miscalculation on their part than the grand expression of market dynamics that I asserted. They're course-correcting on this and are planning to add more warmth back to its responses.
georgemcbay 1y ago
Any of the LLMs (including google's own) is a better "google" than traditional google search. Not really for any technical reason as much as it was that google was perfectly willing to cede the search war to garbage SEO sites as long as they were driving eyeballs to adsense. But the rest I agree with.
piva00 1y ago
I wouldn't say a "worse Google", and I'm also tired of the hallucinations and people copy-pasting AI-generated bullshit as if it was correct. It's just a different tool than Google, and quite complimentary in many ways: instead of doing many keywords searches to collect information, aggregate it in a mental model, and derive a conclusion, it's easier to ask it to answer the aggregation I need and back-verify if it makes sense. To me it has helped a lot to not have to know the right incantations of keywords for the information I'm trying to find right out of the bat, and from the LLM answer I can backtrace what searches to do to confirm the validity of it. It's a small twist but it has definitely showed me some value in these tools for more general information lookup. The thing is: I still want and need to have a search engine, if for some reason search engines cease to exist and LLMs take their place I will not be able to trust anything...

aurareturn 1y ago

I agree with your first part. There are a ton of people who want to use LLMs to solve the Riemann hypothesis but those people will need a different model with vastly more compute than just regular old ChatGPT. ASI isn't what people want or not. It's an AGI that is able to self improve at a rapid rate. We haven't built something that can self improve continuously yet. It's not related to whether people want it or not. Personally, I do want ASI.

827a 1y ago
I simply disagree on that specific definition of ASI. Sure, what we're both describing is a system that is able to self-improve at a rapid rate. But, what does "improve" mean? Is an AI which is able to rapidly self-improve only along the intelligence vertical of making the best sourdough bread actually ASI? > We haven't built something that can self improve continuously yet. Actually, we have. Its called AlphaGo; its capable of rapid self-improvement on the intelligence vertical of playing Go. Is that ASI? If your answer to that is No, then you've accepted the base case of a more general idea: That the vertical of self-improvement does matter in the human designation of ASI, which means the objective and reward functions defining "ASI" are at some level human designated. In other words, it is deeply related to what humans want.