9 ms·

I doubt many people believe that LLMs are conscious[0], but if they did—a belief in sentient/conscious LLMs really implies that using LLMs the way we do at scal

I doubt many people believe that LLMs are conscious[0], but if they did—a belief in sentient/conscious LLMs really implies that using LLMs the way we do at scale (doing things equivalent to torture, mass killing, etc.) may qualify as abuse of sentient beings, which would bring down the whole industry. The reason is that there is no working definition of “consciousness” or “sentience” that does not imply “human-like”, which in turn implies ability to feel and suffer, and what we do with LLMs would generally be considered something that would make beings with human-like sentience and consciousness suffer. [0] Some definitely do, though; or at least they behave with LLMs in a way one would behave with a conscious being.

gavinray 9mo ago
I think it's difficult to have a serious discussion about consciousness online because it's such a mushy thing to define. If you follow the line of thinking that consciousness is an emergent phenomenon, arising out of complexity, it doesn't seem far-fetched to me to believe that someday in the future, a silicon-based computing machine (rather than a biological, carbon-based computing machine) might be "conscious" -- whatever that means.
- strogonoff 9mo ago
  > I think it's difficult to have a serious discussion about consciousness online because it's such a mushy thing to define. It’s circular and self-referential. In defining consciousness we, to phrase it in the least nuanced way, are trying to define a thing through which we define things. The best definition we have reduces to something along the lines of “what we, humans, experience”. By its very nature it makes us unable to fathom or even recognize a hypothetical consciousness if it is entirely unlike ours and/or operates on radically different scales; anything we call “conscious” is implied to be human-like.
Kim_Bruning 9mo ago
An LLM will behave the way you treat it, for better or for worse. From an objective, empirical, scientific point of view, consciousness and feelings are not really fantastically defined. But looking at diverse tests that ARE available, modern LLMs seem to get interesting scores on a number of them. The counter-argument being -of course- that no one ever made those tests with LLMs in mind. But that's not something you should come up with post-hoc. Define better experiments instead! (The ethical issues you mention should probably be (re-)evaluated once systems have continuous memory/context)
- strogonoff 9mo ago
  > An LLM will behave the way you treat it, for better or for worse An LLM produces better output if you treat it badly (threaten violence, gaslight, etc.), which is hardly true for a human unless you count humans in slavery-like conditions.
  - Kim_Bruning 9mo ago
    That's generally not what I find. Do you have a source for that?
    - strogonoff 9mo ago
      Sergey Brin dropped it once, for example[0]: > You know, that’s a weird thing, we don’t circulate this much in the AI community… Not just our models, but all models tend to do better if you threaten them. [0] https://au.lifehacker.com/ai/114236/news/googles-co-founder-says-ai-performs-best-when-you-threaten-it https://au.lifehacker.com/ai/114236/news/googles-co-founder-...

gavinray 9mo ago

I think it's difficult to have a serious discussion about consciousness online because it's such a mushy thing to define. If you follow the line of thinking that consciousness is an emergent phenomenon, arising out of complexity, it doesn't seem far-fetched to me to believe that someday in the future, a silicon-based computing machine (rather than a biological, carbon-based computing machine) might be "conscious" -- whatever that means.

strogonoff 9mo ago
> I think it's difficult to have a serious discussion about consciousness online because it's such a mushy thing to define. It’s circular and self-referential. In defining consciousness we, to phrase it in the least nuanced way, are trying to define a thing through which we define things. The best definition we have reduces to something along the lines of “what we, humans, experience”. By its very nature it makes us unable to fathom or even recognize a hypothetical consciousness if it is entirely unlike ours and/or operates on radically different scales; anything we call “conscious” is implied to be human-like.

Kim_Bruning 9mo ago

An LLM will behave the way you treat it, for better or for worse. From an objective, empirical, scientific point of view, consciousness and feelings are not really fantastically defined. But looking at diverse tests that ARE available, modern LLMs seem to get interesting scores on a number of them. The counter-argument being -of course- that no one ever made those tests with LLMs in mind. But that's not something you should come up with post-hoc. Define better experiments instead! (The ethical issues you mention should probably be (re-)evaluated once systems have continuous memory/context)

strogonoff 9mo ago
> An LLM will behave the way you treat it, for better or for worse An LLM produces better output if you treat it badly (threaten violence, gaslight, etc.), which is hardly true for a human unless you count humans in slavery-like conditions.
- Kim_Bruning 9mo ago
  That's generally not what I find. Do you have a source for that?
  - strogonoff 9mo ago
    Sergey Brin dropped it once, for example[0]: > You know, that’s a weird thing, we don’t circulate this much in the AI community… Not just our models, but all models tend to do better if you threaten them. [0] https://au.lifehacker.com/ai/114236/news/googles-co-founder-says-ai-performs-best-when-you-threaten-it https://au.lifehacker.com/ai/114236/news/googles-co-founder-...