5 ms·

> It would've been nice to reserve "AI" for superior human-like intelligence capable of genuine common sense and reasoning. What would a frontier API have to b

> It would've been nice to reserve "AI" for superior human-like intelligence capable of genuine common sense and reasoning. What would a frontier API have to be able to do to satisfy you?

dominotw 3mo ago
Can it produce a chart topping album if its given all the tools and the prompt "produce chart topping album" . you might say almost no humans can do tht either but some human can but no ai can.
Hilliard_Ohiooo 3mo ago
To answer for OP: We are now calling text and image generators "intelligent" in the same way a spell checker is intelligent. Whatever it's become, "AI" research started as a way to study digital neurology, or how to digitize a mind, not just how to generate data. The Turing Test should have had a caveat, it needs to fool a, "non-stupid" person, and we still have not gotten even close to passing that version.
- radial_symmetry 3mo ago
  What exactly would a 'non-stupid' person do to catch the latest models on a Turing Test? Aside from being aware of AI 'tells' like em-dashes.
  - dominotw 3mo ago
    If this was true. I would repalce myself with ai that pretends to be me on slack. my coworkers would know almost immediately if i did that.
    - HeatrayEnjoyer 3mo ago
      > my coworkers would know almost immediately if i did that. The same would happen if you were replaced by any random human.
      - daveguy 3mo ago
        But immitating others is about the only thing genAI does. Sometimes "others" is a 'programmer', sometimes "others" is an 'artist', but regardless, it still does it poorly.
      - dominotw 3mo ago
        that would be a silly test then
        pixl97 3mo ago
        It sounds kind of like you made up a silly test then.
        dominotw 3mo ago
        what test?
jacobgold 3mo ago
Maybe just a very rigorous version of the Turing test? Modern LLMs can superficially simulate conversation but it's trivial to force them into revealing their non-human like intelligence. They've been "patched" since but all models fail basic tests like "Should I walk or drive to the car wash which is 100 feet away" by recommending you walk. So you'd just ask questions that require theory of mind, abstract and common sense reasoning, causal inference, learning novel rules, transferring knowledge novel situations, recognizing ambiguity, etc.
- bgilroy26 3mo ago
  I would walk
  - wx196 3mo ago
    Me too. At least it doesn't say I need to wash my car.
- davidpapermill 3mo ago
  Can you give me one example that works on Claude right now? I'm never sure whether this indicates "no reasoning present" or you've just hit an odd behaviour in the AI such that its reasoning fails. For example, you present a problem in a way that's dissimilar to the way problems are presented in its training set. That doesn't mean it's not reasoning, just it can only reason correctly in some circumstances.
  - jacobgold 3mo ago
    The models are continually patched with training and post-training. All you have to do is find an area they haven't patched yet, and they'll be just as stupid. I run into deep technical examples every day where they fail in the most basic ways no human ever would. I'm pretty sure most people building these models would admit they don't operate as human-like intelligences? It's baffling that anyone thinks they are.
    - 3mo ago
contagiousflow 3mo ago
strawberry
daveguy 3mo ago
Not OP, but I'd settle for something that actually learns, instead of being a static pile of linear algebra. Pretending it learns because you change the input (context) doesn't count.

> It would've been nice to reserve "AI" for superior human-like intelligence capable of genuine common sense and reasoning. What would a frontier API have to be able to do to satisfy you?

dominotw 3mo ago

Can it produce a chart topping album if its given all the tools and the prompt "produce chart topping album" . you might say almost no humans can do tht either but some human can but no ai can.

Hilliard_Ohiooo 3mo ago

To answer for OP: We are now calling text and image generators "intelligent" in the same way a spell checker is intelligent. Whatever it's become, "AI" research started as a way to study digital neurology, or how to digitize a mind, not just how to generate data. The Turing Test should have had a caveat, it needs to fool a, "non-stupid" person, and we still have not gotten even close to passing that version.

radial_symmetry 3mo ago
What exactly would a 'non-stupid' person do to catch the latest models on a Turing Test? Aside from being aware of AI 'tells' like em-dashes.
- dominotw 3mo ago
  If this was true. I would repalce myself with ai that pretends to be me on slack. my coworkers would know almost immediately if i did that.
  - HeatrayEnjoyer 3mo ago
    > my coworkers would know almost immediately if i did that. The same would happen if you were replaced by any random human.
    - daveguy 3mo ago
      But immitating others is about the only thing genAI does. Sometimes "others" is a 'programmer', sometimes "others" is an 'artist', but regardless, it still does it poorly.
    - dominotw 3mo ago
      that would be a silly test then
      - pixl97 3mo ago
        It sounds kind of like you made up a silly test then.
        dominotw 3mo ago
        what test?

jacobgold 3mo ago

Maybe just a very rigorous version of the Turing test? Modern LLMs can superficially simulate conversation but it's trivial to force them into revealing their non-human like intelligence. They've been "patched" since but all models fail basic tests like "Should I walk or drive to the car wash which is 100 feet away" by recommending you walk. So you'd just ask questions that require theory of mind, abstract and common sense reasoning, causal inference, learning novel rules, transferring knowledge novel situations, recognizing ambiguity, etc.

bgilroy26 3mo ago
I would walk
- wx196 3mo ago
  Me too. At least it doesn't say I need to wash my car.
davidpapermill 3mo ago
Can you give me one example that works on Claude right now? I'm never sure whether this indicates "no reasoning present" or you've just hit an odd behaviour in the AI such that its reasoning fails. For example, you present a problem in a way that's dissimilar to the way problems are presented in its training set. That doesn't mean it's not reasoning, just it can only reason correctly in some circumstances.
- jacobgold 3mo ago
  The models are continually patched with training and post-training. All you have to do is find an area they haven't patched yet, and they'll be just as stupid. I run into deep technical examples every day where they fail in the most basic ways no human ever would. I'm pretty sure most people building these models would admit they don't operate as human-like intelligences? It's baffling that anyone thinks they are.
  - 3mo ago

contagiousflow 3mo ago

strawberry

daveguy 3mo ago

Not OP, but I'd settle for something that actually learns, instead of being a static pile of linear algebra. Pretending it learns because you change the input (context) doesn't count.