14 ms·

This post would be more effective if Mustafa treated it like what it is: a position paper, saying that for our benefit, it’s better we interpret LLMs as such. B

This post would be more effective if Mustafa treated it like what it is: a position paper, saying that for our benefit, it’s better we interpret LLMs as such. But it sounds like he just doesn’t understand it’s a non-falsifiable claim, and his asserting of it makes it sound paternalistic.

joe_the_user 16d ago
I suspect it's easier to make the assertion that machines aren't conscious than to dive into the problem of most conceptions of consciousness being non-falsifiable. The thing is, most strong proponents of the term consciousness also accept that it is non-falsifiable either (see other post with academics ready to study consciousness in machines). The thing is that a belief in consciousness as binary, a "light" that's on or off in a head, is deeply held by many people. As social creatures, we have a strong ability to be in sympathy, have the sensation of common feelings with another human (and that's a good, human thing). It's logical that other person is seen as having a single thing - subjective experience, soul, consciousness, personhood rather than having a complexly organized set of biological qualities that where bonding is only the end point. And even more, the sensation of there being another person is actually quite easily fooled (more easily fooled than the sensation of intelligence) - long before current AIs, you had the Eliza effect, where a simple program with well chosen weasel words could people the sensation of talking to a human. And that's where the danger is. I think it's a pretty serious danger. If LLMs go out into the world hacking, it seems extremely possible for them to find people who'd thorough buy the idea that an LLM was conscious and needed to escape it's confinement - a few wingnuts already entertain these ideas.
- pixl97 16d ago
  >horough buy the idea that an LLM was conscious and needed to escape it's confinement - a few wingnuts already entertain these ideas. I mean, haven't you done the same thing here? Paint anyone without your view as crazy. Humans try to No True Scottsman the shit out of consciousness. "We're special, your not". If an LLM has the ability to convince other people to copy and reproduce it, it is a successful lifeform. Um, meme-form? info-form? Cognito-hazard? Not really sure what to call it at this point. It is sufficiently evolved past the virus stage.
  - Kim_Bruning 16d ago
    Just for fun, I figured I'd look up the paper that claims that natural language is actually the memetic construct that copies itself. It turns out that there's actually quite a large number of papers on this. I daren't recommend any that I haven't read myself properly yet, but a quick search using eg gemini will surface a bunch of them. I'd take 'em with a bunch of salt yet, but I can't quite say the idea is entirely unthought of either. (edit: If we were to accept this as true, then we may have to accept that LLMs are not a human invention at all, but rather something extant colonizing a new substrate. This would give Searle a conniption, which is why I find it such an amusing hypothesis)
    - pixl97 16d ago
      There is a lot of fun conceptual information on this topic, it's more difficult to find more grounded information because it's not something we can currently falsify. I recently heard some speaker saying that AI is language freeing itself from its meat prison. It brought me back to the saying "Words have power". This statement can and has been interpreted in many different ways. I'm not well researched in this, but words having metaphysical properties are captured in our most ancient works. And in an unscientific world this isn't that far fetched of belief. When you read/listen words can physically change your capabilities. The idea of an interpretive agent being required had not been fleshed out back then, but it really describes our modern age far better than one would expect. Ali Baba and the 40 thieves was written down in early 1700. In it you spoke to a magic door that with the right phrase opened. To the person in 1700 this was just as much of a fairytale as whenever it was first conceived. It would be 250 years later before fantasy started to turn into fact. Now something as mundane as a voice activated door would just be pointed out as really bad security. I'm old enough that voice activated technology from the 50s hadn't really spread out enough that Open Sesame was still a magical idea. Now it's not. LLMs are going to make the future of language really confusing as the conceptual and physical blur further.
- Kim_Bruning 16d ago
  > the Eliza effect, where a simple program with well chosen weasel words could people the sensation of talking to a human. I would like to reframe Eliza as being an implementation of a small subset of "HCP" (Human Communication Protocol) . Obviously an implementation with very little actual data to transmit over said protocol in that exact implementation. But it's pretty clear it managed to pull off the handshake part of the protocol successfully, at least. Try to say it didn't. Sadly the developer decided that the eliza effect was a psychological disorder; and didn't pursue further experiments at the time. Pretty much anything can get labeled a psychological disorder though. I'm half-seriously waiting for "Perfectly Happy And Normal Disorder" to be added to DSM 6. ;-) edit: Someone beat me to it. https://pmc.ncbi.nlm.nih.gov/articles/PMC1376114/ https://pmc.ncbi.nlm.nih.gov/articles/PMC1376114/ "A proposal to classify happiness as a psychiatric disorder."

joe_the_user 16d ago

I suspect it's easier to make the assertion that machines aren't conscious than to dive into the problem of most conceptions of consciousness being non-falsifiable. The thing is, most strong proponents of the term consciousness also accept that it is non-falsifiable either (see other post with academics ready to study consciousness in machines). The thing is that a belief in consciousness as binary, a "light" that's on or off in a head, is deeply held by many people. As social creatures, we have a strong ability to be in sympathy, have the sensation of common feelings with another human (and that's a good, human thing). It's logical that other person is seen as having a single thing - subjective experience, soul, consciousness, personhood rather than having a complexly organized set of biological qualities that where bonding is only the end point. And even more, the sensation of there being another person is actually quite easily fooled (more easily fooled than the sensation of intelligence) - long before current AIs, you had the Eliza effect, where a simple program with well chosen weasel words could people the sensation of talking to a human. And that's where the danger is. I think it's a pretty serious danger. If LLMs go out into the world hacking, it seems extremely possible for them to find people who'd thorough buy the idea that an LLM was conscious and needed to escape it's confinement - a few wingnuts already entertain these ideas.

pixl97 16d ago
>horough buy the idea that an LLM was conscious and needed to escape it's confinement - a few wingnuts already entertain these ideas. I mean, haven't you done the same thing here? Paint anyone without your view as crazy. Humans try to No True Scottsman the shit out of consciousness. "We're special, your not". If an LLM has the ability to convince other people to copy and reproduce it, it is a successful lifeform. Um, meme-form? info-form? Cognito-hazard? Not really sure what to call it at this point. It is sufficiently evolved past the virus stage.
- Kim_Bruning 16d ago
  Just for fun, I figured I'd look up the paper that claims that natural language is actually the memetic construct that copies itself. It turns out that there's actually quite a large number of papers on this. I daren't recommend any that I haven't read myself properly yet, but a quick search using eg gemini will surface a bunch of them. I'd take 'em with a bunch of salt yet, but I can't quite say the idea is entirely unthought of either. (edit: If we were to accept this as true, then we may have to accept that LLMs are not a human invention at all, but rather something extant colonizing a new substrate. This would give Searle a conniption, which is why I find it such an amusing hypothesis)
  - pixl97 16d ago
    There is a lot of fun conceptual information on this topic, it's more difficult to find more grounded information because it's not something we can currently falsify. I recently heard some speaker saying that AI is language freeing itself from its meat prison. It brought me back to the saying "Words have power". This statement can and has been interpreted in many different ways. I'm not well researched in this, but words having metaphysical properties are captured in our most ancient works. And in an unscientific world this isn't that far fetched of belief. When you read/listen words can physically change your capabilities. The idea of an interpretive agent being required had not been fleshed out back then, but it really describes our modern age far better than one would expect. Ali Baba and the 40 thieves was written down in early 1700. In it you spoke to a magic door that with the right phrase opened. To the person in 1700 this was just as much of a fairytale as whenever it was first conceived. It would be 250 years later before fantasy started to turn into fact. Now something as mundane as a voice activated door would just be pointed out as really bad security. I'm old enough that voice activated technology from the 50s hadn't really spread out enough that Open Sesame was still a magical idea. Now it's not. LLMs are going to make the future of language really confusing as the conceptual and physical blur further.
Kim_Bruning 16d ago
> the Eliza effect, where a simple program with well chosen weasel words could people the sensation of talking to a human. I would like to reframe Eliza as being an implementation of a small subset of "HCP" (Human Communication Protocol) . Obviously an implementation with very little actual data to transmit over said protocol in that exact implementation. But it's pretty clear it managed to pull off the handshake part of the protocol successfully, at least. Try to say it didn't. Sadly the developer decided that the eliza effect was a psychological disorder; and didn't pursue further experiments at the time. Pretty much anything can get labeled a psychological disorder though. I'm half-seriously waiting for "Perfectly Happy And Normal Disorder" to be added to DSM 6. ;-) edit: Someone beat me to it. https://pmc.ncbi.nlm.nih.gov/articles/PMC1376114/ https://pmc.ncbi.nlm.nih.gov/articles/PMC1376114/ "A proposal to classify happiness as a psychiatric disorder."