4 ms·

Doesn't this ignore the possibility of emergent behavior? We're just a bag of atoms bumping around, and yet we don't dismiss our intelligence.

by ijidak 1mo ago

applfanboysbgon 1mo ago
Please give me a break with this tired trope. Every single fucking time. I am not commenting on the possibility of machine consciousness in general. It may be possible! But there is absolutely zero evidence suggesting language models have it. This idea that this trivial shitty little class of programs we've created are somehow as complex as our biology is ridiculous. There is "emergent behaviour" in the same way that the Game of Life has emergent behaviour. There are solutions to problems, some humans haven't solved before, in the same way that Chess engines have solved Chess far beyond what humans are capable of. Nothing we haven't seen from software before. Software is extremely useful, after all. But the hubris to think we've reached the pinnacle, that there is no further development left, that humanity has become God and solved consciousness, because we programmed software that can convincingly generate strings of words that mimick our language. It's just fundamentally preposterous. Especially if you spend any amount of time actually programming them yourself, it becomes increasingly hard to entertain such ridiculous notions unless you're enticed with bags of money to deceive people into believing things about your software that aren't true.
- mitchdoogle 1mo ago
  Honestly, this seems like a personal pet peeve of yours. You have a bias against machines and place biological processes on a pedestal where they don't belong.
- optimalsolver 1mo ago
  I don't know about phenomenal consciousness, but the following study certainly suggests they have access consciousness: https://www.anthropic.com/research/global-workspace https://www.anthropic.com/research/global-workspace
  - vhantz 1mo ago
    > study > anthropic.com link Sure
- hax0ron3 1mo ago
  What kind of solid evidence could there possibly be that anything other than myself (or, for you, yourself) has consciousness? I believe that other people have consciousness, I feel other people's consciousness strongly and directly, when I look into someone's eyes I feel that I am in the presence of consciousness. But none of these things really add up to the kind of evidence that science usually takes as trustworthy.
  - applfanboysbgon 1mo ago
    We don't need to muddy the issue. Consciousness is a muddy issue, there is no clear answer. But there is a clear answer as to what is not conscious. Nobody asks if a rock is conscious. Nobody asks if a calculator is conscious. Nobody asks if Stockfish is conscious. But make your program generate a few sentences based on statistics and hey, now people won't shut the fuck up about consciousness because magical thinking is more fun than understanding how technology works.
    - Kim_Bruning 1mo ago
      Apropos magical thinking vs understanding, please predict the next token(s) in the following exchange, and then explain how it is arrived at by an opus-level LLM or better. 'What is 158395023132+20403412121?' (I picked a large number of digits to make it unlikely for this exact sum to be in the training set)
      - applfanboysbgon 1mo ago
        Prediction, by definition, can extend beyond what has been literally seen in the training data. With the tokens "2 + 2 = ", the overwhelming prediction is going to be 4, but with enough samples, you can generalise the prediction to apply to more numbers. However, that is all that it is - a prediction. Humans are capable of engaging in prediction, using heuristics as a method of conserving mental energy, because always engaging in full logical reasoning would be a waste of the body's resources. However, humans can also follow a set of logical rules and arrive at their conclusion deterministically, something which is completely outside of an LLM's programming. I don't really care to publicly write about my tests because they will become training targets and not be usable for future internet arguments anyways, but there are a great number of trivial 2~3 sentence logical prompts that will completely fuck an LLM's prediction algorithm and result in incoherent replies that a human, or really anything with a theory of mind, would never generate. Not that a human would always answer correctly on the first try, but the failure methods happen to be completely different, eg. Sol will short-circuit and repeat the prompt verbatim (when the instructions don't remotely suggest doing anything of that nature), even on Max. Prediction can superficially resemble reasoning when there's sufficient training data, but it breaks down severely when confronting a task that is OoD.
- Kim_Bruning 1mo ago
  > This idea that this trivial shitty little class of programs we've created are somehow as complex as our biology is ridiculous. I think you might have a core of truth there. I'd argue that LLMs run natural language. As the name suggests, natural language is not something that humans have artificially architected.
RandomLensman 1mo ago
Not the OP, but complex emergent behavior and intelligence don't need to go together. Understanding what happened might not need intelligence in the mix when a large number of machines with some randomness interact a lot.
- pixl97 1mo ago
  >complex emergent behavior and intelligence don't need to go together More of complex emergent behavior doesn't mean whatever it is, is intelligent. But, if something is intelligent, it will have complex emergent behavior. Of course another problem you're going to have here is defining intelligence as some definitions of it would include a lot of complex emergent behavior.
jurgenburgen 1mo ago
Do you believe ants have consciousness?
- black_knight 1mo ago
  According to Hofstadter, it is not the ants themselves we should say are conscious, but the anthill. And it might very well befriend the anteater eating its ants.
- danielbln 1mo ago
  Not more than a bunch of neurons have conciousness. But put them together and wire them up just right..
gps372 1mo ago
This is why the theory of us 'just' being a bag of atoms doesn't add up. This theory doesn't differentiate 'us' from a furniture where we easily dismiss it's intelligence.
- pixl97 1mo ago
  To have intelligence you have to process information, the question is when does processing information turn into intelligence.
  - gps372 1mo ago
    Information processing might be part of intelligence cycle. I don't think there should be a presumption that intelligence is part of information processing system itself, it could be outside the information processing system as well. For example computers have been processing information for decades now, but only recently they have been almost-successfully accused of having (artificial) intelligence. Intelligence was applied while writing the programs which process information (outside or before system even boots up), not while executing those programs. Its only now with LLMs (and agents based on LLMs) we are noticing that systems can do intent extraction, intent management, and ReAct to it.
joshheitzman 1mo ago
Emergent behavior is not new in the realm of software. Cellular automata has emergent behavior that can get pretty wild. For example: https://en.wikipedia.org/wiki/Lenia https://en.wikipedia.org/wiki/Lenia There are huge differences between brains and computers running LLMs. One of the big ones is that we (collectively) understand how every component of computers running LLMs actually work. The same is not true for neurons.

Doesn't this ignore the possibility of emergent behavior? We're just a bag of atoms bumping around, and yet we don't dismiss our intelligence.

by ijidak 1mo ago

Doesn't this ignore the possibility of emergent behavior? We're just a bag of atoms bumping around, and yet we don't dismiss our intelligence.

applfanboysbgon 1mo ago

Please give me a break with this tired trope. Every single fucking time. I am not commenting on the possibility of machine consciousness in general. It may be possible! But there is absolutely zero evidence suggesting language models have it. This idea that this trivial shitty little class of programs we've created are somehow as complex as our biology is ridiculous. There is "emergent behaviour" in the same way that the Game of Life has emergent behaviour. There are solutions to problems, some humans haven't solved before, in the same way that Chess engines have solved Chess far beyond what humans are capable of. Nothing we haven't seen from software before. Software is extremely useful, after all. But the hubris to think we've reached the pinnacle, that there is no further development left, that humanity has become God and solved consciousness, because we programmed software that can convincingly generate strings of words that mimick our language. It's just fundamentally preposterous. Especially if you spend any amount of time actually programming them yourself, it becomes increasingly hard to entertain such ridiculous notions unless you're enticed with bags of money to deceive people into believing things about your software that aren't true.

mitchdoogle 1mo ago
Honestly, this seems like a personal pet peeve of yours. You have a bias against machines and place biological processes on a pedestal where they don't belong.
optimalsolver 1mo ago
I don't know about phenomenal consciousness, but the following study certainly suggests they have access consciousness: https://www.anthropic.com/research/global-workspace https://www.anthropic.com/research/global-workspace
- vhantz 1mo ago
  > study > anthropic.com link Sure
hax0ron3 1mo ago
What kind of solid evidence could there possibly be that anything other than myself (or, for you, yourself) has consciousness? I believe that other people have consciousness, I feel other people's consciousness strongly and directly, when I look into someone's eyes I feel that I am in the presence of consciousness. But none of these things really add up to the kind of evidence that science usually takes as trustworthy.
- applfanboysbgon 1mo ago
  We don't need to muddy the issue. Consciousness is a muddy issue, there is no clear answer. But there is a clear answer as to what is not conscious. Nobody asks if a rock is conscious. Nobody asks if a calculator is conscious. Nobody asks if Stockfish is conscious. But make your program generate a few sentences based on statistics and hey, now people won't shut the fuck up about consciousness because magical thinking is more fun than understanding how technology works.
  - Kim_Bruning 1mo ago
    Apropos magical thinking vs understanding, please predict the next token(s) in the following exchange, and then explain how it is arrived at by an opus-level LLM or better. 'What is 158395023132+20403412121?' (I picked a large number of digits to make it unlikely for this exact sum to be in the training set)
    - applfanboysbgon 1mo ago
      Prediction, by definition, can extend beyond what has been literally seen in the training data. With the tokens "2 + 2 = ", the overwhelming prediction is going to be 4, but with enough samples, you can generalise the prediction to apply to more numbers. However, that is all that it is - a prediction. Humans are capable of engaging in prediction, using heuristics as a method of conserving mental energy, because always engaging in full logical reasoning would be a waste of the body's resources. However, humans can also follow a set of logical rules and arrive at their conclusion deterministically, something which is completely outside of an LLM's programming. I don't really care to publicly write about my tests because they will become training targets and not be usable for future internet arguments anyways, but there are a great number of trivial 2~3 sentence logical prompts that will completely fuck an LLM's prediction algorithm and result in incoherent replies that a human, or really anything with a theory of mind, would never generate. Not that a human would always answer correctly on the first try, but the failure methods happen to be completely different, eg. Sol will short-circuit and repeat the prompt verbatim (when the instructions don't remotely suggest doing anything of that nature), even on Max. Prediction can superficially resemble reasoning when there's sufficient training data, but it breaks down severely when confronting a task that is OoD.
Kim_Bruning 1mo ago
> This idea that this trivial shitty little class of programs we've created are somehow as complex as our biology is ridiculous. I think you might have a core of truth there. I'd argue that LLMs run natural language. As the name suggests, natural language is not something that humans have artificially architected.

RandomLensman 1mo ago

Not the OP, but complex emergent behavior and intelligence don't need to go together. Understanding what happened might not need intelligence in the mix when a large number of machines with some randomness interact a lot.

pixl97 1mo ago
>complex emergent behavior and intelligence don't need to go together More of complex emergent behavior doesn't mean whatever it is, is intelligent. But, if something is intelligent, it will have complex emergent behavior. Of course another problem you're going to have here is defining intelligence as some definitions of it would include a lot of complex emergent behavior.

jurgenburgen 1mo ago

Do you believe ants have consciousness?

black_knight 1mo ago
According to Hofstadter, it is not the ants themselves we should say are conscious, but the anthill. And it might very well befriend the anteater eating its ants.
danielbln 1mo ago
Not more than a bunch of neurons have conciousness. But put them together and wire them up just right..

gps372 1mo ago

This is why the theory of us 'just' being a bag of atoms doesn't add up. This theory doesn't differentiate 'us' from a furniture where we easily dismiss it's intelligence.

pixl97 1mo ago
To have intelligence you have to process information, the question is when does processing information turn into intelligence.
- gps372 1mo ago
  Information processing might be part of intelligence cycle. I don't think there should be a presumption that intelligence is part of information processing system itself, it could be outside the information processing system as well. For example computers have been processing information for decades now, but only recently they have been almost-successfully accused of having (artificial) intelligence. Intelligence was applied while writing the programs which process information (outside or before system even boots up), not while executing those programs. Its only now with LLMs (and agents based on LLMs) we are noticing that systems can do intent extraction, intent management, and ReAct to it.

joshheitzman 1mo ago

Emergent behavior is not new in the realm of software. Cellular automata has emergent behavior that can get pretty wild. For example: https://en.wikipedia.org/wiki/Lenia https://en.wikipedia.org/wiki/Lenia There are huge differences between brains and computers running LLMs. One of the big ones is that we (collectively) understand how every component of computers running LLMs actually work. The same is not true for neurons.