3 ms·

Two strikes against the hypothesis: - LLMs suggest that human-like "cognition" can be somewhat mimicked by purely mechanical means. - Exposing the brain to an

by copperx 2mo ago

Two strikes against the hypothesis: - LLMs suggest that human-like "cognition" can be somewhat mimicked by purely mechanical means. - Exposing the brain to an MRI would affect cognition if there was a quantum element to it.

beardyw 2mo ago
Everything is subject to "quantum effects". Not necessarily trying to squeeze superposition and entanglement into a pint pot.
ck2 2mo ago
if you mean LLM seems like cognition LLM emulate cognition, there is a huge difference, it's a mimick process not genuine if you read about psychopaths/sociopaths, they do not comprehend emotions but they can learn to emulate them by watching others and mimicking them that's exactly what LLM is doing, it is taught to assemble words in a way that mimics all the humans in its huge datapool, so it comes off as "cognition" because that's what the humans it's emulating were doing
- jacinabox 2mo ago
  "Feign vulnerability" -GPT4 in one of its many glitches
- j-pb 2mo ago
  Sociopaths can feel emotions, they just lack empathy. So they can consciously manipulate others. LLMs don't have conscious processes that perform that kind of reasoning. If anything the fact that they are blackboxes to their own behavior and are able to non-verbally empathise, is the opposite of what you describe. Your requirment for something supernatural/quantum to decide between emulation and real thing is self referential.
  - ben_w 2mo ago
    > LLMs don't have conscious processes that perform that kind of reasoning. If only we could tell either way, we'd have made a huge breakthrough in the philosophy of consciousness.
    - j-pb 2mo ago
      There is no hidden inner monologue. Human inner monologues produce measurable micoactivations that are the same as the ones produced in speech. If it's not in the inner COT it isn't there.
      - ben_w 2mo ago
        I don't see how this adds much to either a pro or con argument. An inner monologue isn't something all humans have[0]. It is unclear to me how well (or poorly) Chain of Thought mimics intrapersonal communication. J-space probes (and other things) suggest there are, in fact, other aspects to LLM "thought" besides a CoT scratchpad[1]. However, regardless of how those questions pan out, qualia (of emotions in this case, but the problem is in general) is its own mystery, one for which we are still, so far as I can tell, unable to make progress with. We kinda know what emotions are for, what function they serve in our genetic fitness, but why do they feel like anything? We do not know, they just do. [0] https://science.howstuffworks.com/life/inside-the-mind/human-brain/inner-voice.htm https://science.howstuffworks.com/life/inside-the-mind/human... and https://en.wikipedia.org/wiki/Intrapersonal_communication https://en.wikipedia.org/wiki/Intrapersonal_communication [1] https://www.anthropic.com/research/global-workspace https://www.anthropic.com/research/global-workspace
ck2 2mo ago
oh and to address your second idea about MRI when mice were exposed to cosmic ray bursts (to test for effects on past and future astronauts traveling to moon and mars) the mice learned much much more slowly and forgot things much much faster how's that for "cognitive corruption" ? (btw this is also why I laugh when Musk thinks he's going to Mars ever, it's not just a one-way trip, he wouldn't even survive the journey itself with current technology and unlike earth there's no magnetic field or enough atmosphere on Mars to deflect them)
- ben_w 2mo ago
  > unlike earth there's no magnetic field or enough atmosphere on Mars to deflect them These are solvable with known methods. They make the whole thing rather less romantic. If you're going to the trouble of e.g. burying your base under regolith or wrapping a planet in a big coil and lots of PV to make an artificial magnetic field, it very quickly becomes apparent that for essentially all possible disasters short of "oops we made a black hole and it caused a magnitude 15 earthquake and all the crust everywhere liquified and mixed with all the water so even ocean bases weren't safe", it's much, much easier to survive disasters on Earth without actually leaving Earth, than to go to Mars and have a normal day. If someone made a robot factory that went mad and converted all the nitrogen in our atmosphere into ammonia (killing all the plants that no longer had access to the atmospheric nitrogen), while also producing enough smog to halve how much sunlight we got, and dumped a lot of endocrine disruptors into our water cycle, the top of Mount Everest and the Sahara desert would each still be much more hospitable than Mars is today.
  - UltraSane 2mo ago
    It shouldn't be a surprise that it is very hard to keep humans alive outside the exact environment we evolved in. Current squishy humans will never colonize space, only intelligences using a vastly more durable substrate will be able to.
    - ben_w 2mo ago
      I suspect so, but I wouldn't want to bet on it. Enough humans seem to be repelled by the idea of transhumanism that they may well insist on doing it the hard way.
jebarker 2mo ago
> LLMs suggest that human-like "cognition" can be somewhat mimicked by purely mechanical means. This is not a strike against that hypothesis. Quantum systems can be simulated using non-quantum computers but that doesn't make non-quantum computers into quantum computers. Similarly, mimicking some aspects of human cognition doesn't mean you can replicate all of human cognition with the same method.

by copperx 2mo ago

beardyw 2mo ago

Everything is subject to "quantum effects". Not necessarily trying to squeeze superposition and entanglement into a pint pot.

ck2 2mo ago

if you mean LLM seems like cognition LLM emulate cognition, there is a huge difference, it's a mimick process not genuine if you read about psychopaths/sociopaths, they do not comprehend emotions but they can learn to emulate them by watching others and mimicking them that's exactly what LLM is doing, it is taught to assemble words in a way that mimics all the humans in its huge datapool, so it comes off as "cognition" because that's what the humans it's emulating were doing

jacinabox 2mo ago
"Feign vulnerability" -GPT4 in one of its many glitches
j-pb 2mo ago
Sociopaths can feel emotions, they just lack empathy. So they can consciously manipulate others. LLMs don't have conscious processes that perform that kind of reasoning. If anything the fact that they are blackboxes to their own behavior and are able to non-verbally empathise, is the opposite of what you describe. Your requirment for something supernatural/quantum to decide between emulation and real thing is self referential.
- ben_w 2mo ago
  > LLMs don't have conscious processes that perform that kind of reasoning. If only we could tell either way, we'd have made a huge breakthrough in the philosophy of consciousness.
  - j-pb 2mo ago
    There is no hidden inner monologue. Human inner monologues produce measurable micoactivations that are the same as the ones produced in speech. If it's not in the inner COT it isn't there.
    - ben_w 2mo ago
      I don't see how this adds much to either a pro or con argument. An inner monologue isn't something all humans have[0]. It is unclear to me how well (or poorly) Chain of Thought mimics intrapersonal communication. J-space probes (and other things) suggest there are, in fact, other aspects to LLM "thought" besides a CoT scratchpad[1]. However, regardless of how those questions pan out, qualia (of emotions in this case, but the problem is in general) is its own mystery, one for which we are still, so far as I can tell, unable to make progress with. We kinda know what emotions are for, what function they serve in our genetic fitness, but why do they feel like anything? We do not know, they just do. [0] https://science.howstuffworks.com/life/inside-the-mind/human-brain/inner-voice.htm https://science.howstuffworks.com/life/inside-the-mind/human... and https://en.wikipedia.org/wiki/Intrapersonal_communication https://en.wikipedia.org/wiki/Intrapersonal_communication [1] https://www.anthropic.com/research/global-workspace https://www.anthropic.com/research/global-workspace

ck2 2mo ago

oh and to address your second idea about MRI when mice were exposed to cosmic ray bursts (to test for effects on past and future astronauts traveling to moon and mars) the mice learned much much more slowly and forgot things much much faster how's that for "cognitive corruption" ? (btw this is also why I laugh when Musk thinks he's going to Mars ever, it's not just a one-way trip, he wouldn't even survive the journey itself with current technology and unlike earth there's no magnetic field or enough atmosphere on Mars to deflect them)

ben_w 2mo ago
> unlike earth there's no magnetic field or enough atmosphere on Mars to deflect them These are solvable with known methods. They make the whole thing rather less romantic. If you're going to the trouble of e.g. burying your base under regolith or wrapping a planet in a big coil and lots of PV to make an artificial magnetic field, it very quickly becomes apparent that for essentially all possible disasters short of "oops we made a black hole and it caused a magnitude 15 earthquake and all the crust everywhere liquified and mixed with all the water so even ocean bases weren't safe", it's much, much easier to survive disasters on Earth without actually leaving Earth, than to go to Mars and have a normal day. If someone made a robot factory that went mad and converted all the nitrogen in our atmosphere into ammonia (killing all the plants that no longer had access to the atmospheric nitrogen), while also producing enough smog to halve how much sunlight we got, and dumped a lot of endocrine disruptors into our water cycle, the top of Mount Everest and the Sahara desert would each still be much more hospitable than Mars is today.
- UltraSane 2mo ago
  It shouldn't be a surprise that it is very hard to keep humans alive outside the exact environment we evolved in. Current squishy humans will never colonize space, only intelligences using a vastly more durable substrate will be able to.
  - ben_w 2mo ago
    I suspect so, but I wouldn't want to bet on it. Enough humans seem to be repelled by the idea of transhumanism that they may well insist on doing it the hard way.

jebarker 2mo ago

> LLMs suggest that human-like "cognition" can be somewhat mimicked by purely mechanical means. This is not a strike against that hypothesis. Quantum systems can be simulated using non-quantum computers but that doesn't make non-quantum computers into quantum computers. Similarly, mimicking some aspects of human cognition doesn't mean you can replicate all of human cognition with the same method.