6 ms·

> 1. Humans have trouble with complex problems and memory demands. True! But incomplete. We have every right to expect machines to do things we can’t. [...] If

> 1. Humans have trouble with complex problems and memory demands. True! But incomplete. We have every right to expect machines to do things we can’t. [...] If we want to get to AGI, we will have to better. I don't get this argument. The paper is about "whether RLLMs can think". If we grant "humans make these mistakes too", but also "we still require this ability in our definition of thinking", aren't we saying "thinking in humans is a illusion" too?

autobodie 1y ago
Agree. Both sides of the argument are unsatisfying. They seem like quantitative answers to a qualitative question.
- serbuvlad 1y ago
  "Have we created machines that can do something qualitatevely similar to that part of us that can correlate known information and pattern recognition to produce new ideas and solutions to problems -- that part we call thinking?" I think the answer to this question is certainly "Yes". I think the reason people deny this is because it was just laughably easy in retrospect. In mid-2022 people were like. "Wow this GPT3 thing generates kind of coherent greentexts" Since then really only we got: larger models, larger models, search, agents, larger models, chain-of-thought and larger models. And from a novelty toy we got a set of tools that at the very least massively increase human productivity in a wide range of tasks and certainly pass any Turing test. Attention really was all you needed. But of course, if you ask a buddhist monk, he'll tell you we are attention machines, not computation machines. He'll also tell you, should you listen, that we have a monkey in our mind that is constantly producing new thoughts. This monkey is not who we are, it's an organ. It's thoughts are not our thoughts. It's something we perceive. And that we shouldn't identify with. Now we have thought-genrating-monkeys with jet engines and adrenaline shots. This can be good. Thought-genrating-monkeys put us on the moon and wrote Hamlet and the Oddesy. The key is to not become a slave to them. To realize that our worth consists not in our ability to think. And that we are more than that.
  - autobodie 1y ago
    > The key is to not become a slave to them. To realize that our worth consists not in our ability to think. And that we are more than that. I cannot afford to consider whether you are right because I am a slave to capital, and therefore may as well be a slave to capital's LLMs. The same goes for you.
    - serbuvlad 1y ago
      I am not a slave to capital. I am a slave to the harsh nature of the world. I get too hot in summer and too cold in winter. I die of hunger. I am harassed by critters of all sorts. And when my bed breaks, to keep my fragile spine from straining at night, I _want_ some trees to be cut, some mattresses to be provisioned, some designers to be provisioned etc. And capital is what gets me that, from people I will never meet, who wouldn't blink once if I died tomorrow.
      - LinXitoW 1y ago
        Considering capitalism is a very new phenomenon in human history, how do you think people survived and thrived for the other 248000 years? It's as ludicrous to believe that capitalism is some kind of force of nature as it is to believe kings were chosen by god.
        serbuvlad 1y ago
        That depends on how you define your terms. A pro-capital laissez-faire policy is new, sure. But the first civilizations in the world around 3000BC had trade, money, banking, capital accumulation, divison of labour etc.
        TeMPOraL 1y ago
        > how do you think people survived and thrived for the other 248000 years? In small tribes, where everyone knew everyone intimately because they lived together, and everything was managed by feels. Things like rules, laws, money, banking, hierarchies, well-defined private vs. public ownership, are all things that came with scale, because interpersonal relationships fail to keep group cohesion once it reaches more than ~100 people.
FINDarkside 1y ago
Agreed. But also his point about AGI is incorrect. AI that will perform on the level of average human in every task is AGI by definition.
- deleted 1y ago
  [deleted]
- simonw 1y ago
  That very much depends on which AGI definition you are using. I imagine there are a dozen or so variants out there. See also "AI" and "agents" and (apparently) "vibe coding" and pretty much every other piece of jargon in this field.
  - FINDarkside 1y ago
    I think it's very widely accepted definition and there's really no competing definitions either as far as I know. While some people might think AGI means superintelligence, it's only because they've heard the term but never bothered to look up what it means.
    - simonw 1y ago
      OpenAI: https://openai.com/index/how-should-ai-systems-behave/#citation-bottom-A https://openai.com/index/how-should-ai-systems-behave/#citat... "By AGI, we mean highly autonomous systems that outperform humans at most economically valuable work." AWS: https://aws.amazon.com/what-is/artificial-general-intelligence/ https://aws.amazon.com/what-is/artificial-general-intelligen... "Artificial general intelligence (AGI) is a field of theoretical AI research that attempts to create software with human-like intelligence and the ability to self-teach. The aim is for the software to be able to perform tasks that it is not necessarily trained or developed for." DeepMind: https://arxiv.org/abs/2311.02462 https://arxiv.org/abs/2311.02462 "Artificial General Intelligence (AGI) is an important and sometimes controversial concept in computing research, used to describe an AI system that is at least as capable as a human at most tasks. [...] We argue that any definition of AGI should meet the following six criteria: We emphasize the importance of metacognition, and suggest that an AGI benchmark should include metacognitive tasks such as (1) the ability to learn new skills, (2) the ability to know when to ask for help, and (3) social metacognitive abilities such as those relating to theory of mind. The ability to learn new skills (Chollet, 2019) is essential to generality, since it is infeasible for a system to be optimized for all possible use cases a priori [...]" The key difference appears to be around self-teaching and meta-cognition. The OpenAI one shortcuts that by focusing on "outperform humans at most economically valuable work", but others make that ability to self-improve key to their definitions. Note that you said "AI that will perform on the level of average human in every task" - which disagrees very slightly with the OpenAI one (they went with "outperform humans at most economically valuable work"). If you read more of the DeepMind paper it mentions "this definition notably focuses on non-physical tasks", so their version of AGI does not incorporate full robotics.
whatagreatboy 1y ago
the real ability of intelligence is to correct mistakes in a gradual and consistent way.
briandw 1y ago
Humans use tools to extend their abilities. LLM can do the same. In this paper they didn’t allow tool use. When others gave the tower of hanoi task to llms with tool use, like a python env, they were able to complete the task.
- xienze 1y ago
  But the Tower of Hanoi can be solved without "tools" by humans, simply by understanding the problem, thinking about the solution, and writing it out. Having the LLM shell out to a Python example that it "wrote" (or rather, "pasted" since surely a Python solution to the Tower of Hanoi was part of its training set) is akin to a human Googling "program to solve Tower of Hanoi", copy-pasting and running the solution. Yes the LLM has "reasoned" that the solution to the problem is call out to a solution that it "knows" is out there, but that's not really "thinking" about how to solve a problem in the human sense. What happens when some novel Tower of Hanoi-esque puzzle is presented and there's nothing available in its training set to reference as an executable solution? A human can reason about and present a solution, but an LLM? Ehh...
  - DiogenesKynikos 1y ago
    LLMs are perfectly capable of writing code to solve problems that are not in their training set. I ask LLMs to write code for niche problems that you won't find answers to just by Googling all the time. The LLMs usually get it right.
    - xienze 1y ago
      > LLMs are perfectly capable of writing code to solve problems that are not in their training set. Examples of these problems? You'll probably find that they're simply compositions of things already in the training set. For example, you might think that "here's a class containing an ID field and foobar field. Make a linked list class that stores inserted items in reverse foobar order with the ID field breaking ties" is something "not in" the training set, but it's really just a composition of the "make a linked list class" and "sort these things based on a field" problems.
YeGoblynQueenne 1y ago
>> I don't get this argument. The argument is that LLMs are computer systems and a computer system that's as bad as a human is less useful than a human.
tim333 1y ago
To understand the argument you have to allow that the Gary Marcus neural network has a fixed bias towards LLMs being rubbish and so tends to confabulate these sort of things. (Geoffrey Hinton is quite funny on that https://youtu.be/d7ltNiRrDHQ https://youtu.be/d7ltNiRrDHQ)

autobodie 1y ago

Agree. Both sides of the argument are unsatisfying. They seem like quantitative answers to a qualitative question.

serbuvlad 1y ago
"Have we created machines that can do something qualitatevely similar to that part of us that can correlate known information and pattern recognition to produce new ideas and solutions to problems -- that part we call thinking?" I think the answer to this question is certainly "Yes". I think the reason people deny this is because it was just laughably easy in retrospect. In mid-2022 people were like. "Wow this GPT3 thing generates kind of coherent greentexts" Since then really only we got: larger models, larger models, search, agents, larger models, chain-of-thought and larger models. And from a novelty toy we got a set of tools that at the very least massively increase human productivity in a wide range of tasks and certainly pass any Turing test. Attention really was all you needed. But of course, if you ask a buddhist monk, he'll tell you we are attention machines, not computation machines. He'll also tell you, should you listen, that we have a monkey in our mind that is constantly producing new thoughts. This monkey is not who we are, it's an organ. It's thoughts are not our thoughts. It's something we perceive. And that we shouldn't identify with. Now we have thought-genrating-monkeys with jet engines and adrenaline shots. This can be good. Thought-genrating-monkeys put us on the moon and wrote Hamlet and the Oddesy. The key is to not become a slave to them. To realize that our worth consists not in our ability to think. And that we are more than that.
- autobodie 1y ago
  > The key is to not become a slave to them. To realize that our worth consists not in our ability to think. And that we are more than that. I cannot afford to consider whether you are right because I am a slave to capital, and therefore may as well be a slave to capital's LLMs. The same goes for you.
  - serbuvlad 1y ago
    I am not a slave to capital. I am a slave to the harsh nature of the world. I get too hot in summer and too cold in winter. I die of hunger. I am harassed by critters of all sorts. And when my bed breaks, to keep my fragile spine from straining at night, I _want_ some trees to be cut, some mattresses to be provisioned, some designers to be provisioned etc. And capital is what gets me that, from people I will never meet, who wouldn't blink once if I died tomorrow.
    - LinXitoW 1y ago
      Considering capitalism is a very new phenomenon in human history, how do you think people survived and thrived for the other 248000 years? It's as ludicrous to believe that capitalism is some kind of force of nature as it is to believe kings were chosen by god.
      - serbuvlad 1y ago
        That depends on how you define your terms. A pro-capital laissez-faire policy is new, sure. But the first civilizations in the world around 3000BC had trade, money, banking, capital accumulation, divison of labour etc.
      - TeMPOraL 1y ago
        > how do you think people survived and thrived for the other 248000 years? In small tribes, where everyone knew everyone intimately because they lived together, and everything was managed by feels. Things like rules, laws, money, banking, hierarchies, well-defined private vs. public ownership, are all things that came with scale, because interpersonal relationships fail to keep group cohesion once it reaches more than ~100 people.

FINDarkside 1y ago

Agreed. But also his point about AGI is incorrect. AI that will perform on the level of average human in every task is AGI by definition.

deleted 1y ago
[deleted]
simonw 1y ago
That very much depends on which AGI definition you are using. I imagine there are a dozen or so variants out there. See also "AI" and "agents" and (apparently) "vibe coding" and pretty much every other piece of jargon in this field.
- FINDarkside 1y ago
  I think it's very widely accepted definition and there's really no competing definitions either as far as I know. While some people might think AGI means superintelligence, it's only because they've heard the term but never bothered to look up what it means.
  - simonw 1y ago
    OpenAI: https://openai.com/index/how-should-ai-systems-behave/#citation-bottom-A https://openai.com/index/how-should-ai-systems-behave/#citat... "By AGI, we mean highly autonomous systems that outperform humans at most economically valuable work." AWS: https://aws.amazon.com/what-is/artificial-general-intelligence/ https://aws.amazon.com/what-is/artificial-general-intelligen... "Artificial general intelligence (AGI) is a field of theoretical AI research that attempts to create software with human-like intelligence and the ability to self-teach. The aim is for the software to be able to perform tasks that it is not necessarily trained or developed for." DeepMind: https://arxiv.org/abs/2311.02462 https://arxiv.org/abs/2311.02462 "Artificial General Intelligence (AGI) is an important and sometimes controversial concept in computing research, used to describe an AI system that is at least as capable as a human at most tasks. [...] We argue that any definition of AGI should meet the following six criteria: We emphasize the importance of metacognition, and suggest that an AGI benchmark should include metacognitive tasks such as (1) the ability to learn new skills, (2) the ability to know when to ask for help, and (3) social metacognitive abilities such as those relating to theory of mind. The ability to learn new skills (Chollet, 2019) is essential to generality, since it is infeasible for a system to be optimized for all possible use cases a priori [...]" The key difference appears to be around self-teaching and meta-cognition. The OpenAI one shortcuts that by focusing on "outperform humans at most economically valuable work", but others make that ability to self-improve key to their definitions. Note that you said "AI that will perform on the level of average human in every task" - which disagrees very slightly with the OpenAI one (they went with "outperform humans at most economically valuable work"). If you read more of the DeepMind paper it mentions "this definition notably focuses on non-physical tasks", so their version of AGI does not incorporate full robotics.

whatagreatboy 1y ago

the real ability of intelligence is to correct mistakes in a gradual and consistent way.

briandw 1y ago

Humans use tools to extend their abilities. LLM can do the same. In this paper they didn’t allow tool use. When others gave the tower of hanoi task to llms with tool use, like a python env, they were able to complete the task.

xienze 1y ago
But the Tower of Hanoi can be solved without "tools" by humans, simply by understanding the problem, thinking about the solution, and writing it out. Having the LLM shell out to a Python example that it "wrote" (or rather, "pasted" since surely a Python solution to the Tower of Hanoi was part of its training set) is akin to a human Googling "program to solve Tower of Hanoi", copy-pasting and running the solution. Yes the LLM has "reasoned" that the solution to the problem is call out to a solution that it "knows" is out there, but that's not really "thinking" about how to solve a problem in the human sense. What happens when some novel Tower of Hanoi-esque puzzle is presented and there's nothing available in its training set to reference as an executable solution? A human can reason about and present a solution, but an LLM? Ehh...
- DiogenesKynikos 1y ago
  LLMs are perfectly capable of writing code to solve problems that are not in their training set. I ask LLMs to write code for niche problems that you won't find answers to just by Googling all the time. The LLMs usually get it right.
  - xienze 1y ago
    > LLMs are perfectly capable of writing code to solve problems that are not in their training set. Examples of these problems? You'll probably find that they're simply compositions of things already in the training set. For example, you might think that "here's a class containing an ID field and foobar field. Make a linked list class that stores inserted items in reverse foobar order with the ID field breaking ties" is something "not in" the training set, but it's really just a composition of the "make a linked list class" and "sort these things based on a field" problems.

YeGoblynQueenne 1y ago

>> I don't get this argument. The argument is that LLMs are computer systems and a computer system that's as bad as a human is less useful than a human.

tim333 1y ago

To understand the argument you have to allow that the Gary Marcus neural network has a fixed bias towards LLMs being rubbish and so tends to confabulate these sort of things. (Geoffrey Hinton is quite funny on that https://youtu.be/d7ltNiRrDHQ https://youtu.be/d7ltNiRrDHQ)