11 ms·

With the explosion of LLMs we are close to the final thoughts in her chapter: “this planet may be filled with beings who will be blind and deaf as the inmost ro

by pama 2y ago

With the explosion of LLMs we are close to the final thoughts in her chapter: “this planet may be filled with beings who will be blind and deaf as the inmost rock, yet will execute changes as delicate and complicated as those of human language and all the intricate web of what we call its effects, without sensitive impression, without sensitive impulse.”

XorNot 2y ago
The thing which has become very weird for me is watching Star Trek holodeck episodes in the age of LLMs. When I was in high school we considered this absurd writing - obviously they must be sentient...now LLMs have happened and it feels obvious that the opposite can be true. Same story with Star Wars and droids: C3PO wasn't sentient, just a protocol model loaded into a droid frame in the wrong environment, it's attempts to constantly being the conversation back to politeness being a simplistically coded filter layer.
- optimalsolver 2y ago
  Another thing Trek (and other SF) didn't predict was that AI would speak with Bay Area vocal fry. Someone should re-dub the Enterprise computer's lines with it.
  - yapyap 2y ago
    I think that’s just the AI you’re using..
- captainclam 2y ago
  Haha, I've had the same thoughts, that of course computers/AI/droids of that conversational capacity were conscious. You'd be a brute not to think that! And all of a sudden, LLMs absolutely have the command of natural language that once seemed such an obvious indicator of sentience, and now I find myself one of those bigots who don't believe in robot rights! I'm being silly, but I do think there are implications here with respect to the future debate on AI sentience. I guess I once thought there would be this threshold where the reality of an AI's inner experience became blatantly obvious, but I see now that this is going to be a profoundly thorny problem. Who knows, maybe in several decades we'll have a consciousness-o-meter that demonstrates that LLMs have had some degree of awareness all along.
  - kenjackson 2y ago
    What seems far more likely, the way the world is going, is that we’ll use the consciousness-o-meter to determine some groups of humans are less conscious than others.
  - ANewFormation 2y ago
    The fundamental problem is that we keep setting 'reasonable' benchmarks, implicitly because we want to make them seem reachable, but then when we inevitably do we find it feels we [of course] haven't achieved what we really wanted to in the first place. True artificial intelligence would have the ability to meaningfully and significantly improve itself, recursively - a characteristic of even the most primitive life on Earth. And the implications of a computing system capable of such simply can't be overstated. With access to essentially all contemporary knowledge, flawless recall, and stupidly ridiculous amounts of energy to power itself (relative to e.g. a brain), it would create an explosion of knowledge in every single domain imaginable, and many we can't even yet fathom. And it would be doing this at an accelerating pace owing to self improvement. Basically the very nature of existence and knowledge would change. Sentience or not would be irrelevant as we simply tried (and inevitably failed, miserably) to catch up. But this is a bit further out of reach than 'can vaguely pass for a human in a casual non-adversarial q&a style chat.' --- This was one of the rather many areas where Star Trek failed to really consider the implications of its concepts, probably because it would simply break the world building. The Borg, for instance, would be defacto Gods in no time, even without assimilation.
    - readyplayernull 2y ago
      > meaningfully and significantly improve itself Exactly, mechanically by tools, environmentally by science, metaphysically by philosophy. An AI can only become an I by crossing a singularity of knowledge.
    - captainclam 2y ago
      > This was one of the rather many areas where Star Trek failed to really consider the implications of its concepts, probably because it would simply break the world building. This is true of pretty much all scifi! It's funny seeing super-futuristic depictions of star-fighter pilots and combatants with firearms and its just...so crushingly evident that humans will not have supremacy in these arenas very shortly. Frank Herbert must have anticipated this complication and side-stepped the whole issue by preemptively canonizing the Butlerian Jihad.
- jameshart 2y ago
  Threepio reads so much like an LLM, the writing turns out to have been skockingly prescient. At the time of Phantom Menace, the whole story of him being assembled by some slave kid on a backwater planet in the outer rim seemed preposterous - why would a slave kid into pod racing make a pompous ass of a protocol droid? Well, he just installed a standard easily pirated LLM into the droid he made. And the fact that these LLMs are used as 'protocol droids' to communicate with lower-level devices - 'the binary language of moisture vaporators' etc. - matches up with the pattern of LLMs as tools for generating scripts in Python. Terrible though the last sequel is, they even managed to include jailbreaking Threepio to get him to access part of his training data that had been censored out.
nthingtohide 2y ago
Philosophers have coined the term P-zombies. LLMs can be considered as P-zombies if they do not possess consciousness. Functionally equivalent but no soul. Substance Dualism was used by philosophers to deny animals consciousness.
- farts_mckensy 2y ago
  Non materialist bullshit.
- kbelder 2y ago
  >Functionally equivalent but no soul. I think it's an incoherent concept. If the 'soul' is anything, there is some difference in the brain that would allow it. You can't have two people with identical brains and claim that one has a soul, unless you believe in magic.
  - bee_rider 2y ago
    Is there a non-magical description of a soul? I thought it was an essentially magical concept. I’m not sure that makes it incoherent.
  - bobxmax 2y ago
    It's been interesting seeing those with an entirely materialist understanding of the universe try to grapple with and deal with consciousness as LLMs have emerged. To maintain that belief you have to either believe there is a physical source of consciousness, or that consciousness doesn't exist, neither of which most people can accept, yet LLMs are forcing people to confront.
    - thfuran 2y ago
      You think most people can't accept that consciousness is a phenomenon with basis solely in physical reality?
      - Isamu 2y ago
superkuh 2y ago
Basically the plot of Peter Watts' "Blindsight" (post-2000) scifi book series.

by pama 2y ago

XorNot 2y ago

The thing which has become very weird for me is watching Star Trek holodeck episodes in the age of LLMs. When I was in high school we considered this absurd writing - obviously they must be sentient...now LLMs have happened and it feels obvious that the opposite can be true. Same story with Star Wars and droids: C3PO wasn't sentient, just a protocol model loaded into a droid frame in the wrong environment, it's attempts to constantly being the conversation back to politeness being a simplistically coded filter layer.

optimalsolver 2y ago
Another thing Trek (and other SF) didn't predict was that AI would speak with Bay Area vocal fry. Someone should re-dub the Enterprise computer's lines with it.
- yapyap 2y ago
  I think that’s just the AI you’re using..
captainclam 2y ago
Haha, I've had the same thoughts, that of course computers/AI/droids of that conversational capacity were conscious. You'd be a brute not to think that! And all of a sudden, LLMs absolutely have the command of natural language that once seemed such an obvious indicator of sentience, and now I find myself one of those bigots who don't believe in robot rights! I'm being silly, but I do think there are implications here with respect to the future debate on AI sentience. I guess I once thought there would be this threshold where the reality of an AI's inner experience became blatantly obvious, but I see now that this is going to be a profoundly thorny problem. Who knows, maybe in several decades we'll have a consciousness-o-meter that demonstrates that LLMs have had some degree of awareness all along.
- kenjackson 2y ago
  What seems far more likely, the way the world is going, is that we’ll use the consciousness-o-meter to determine some groups of humans are less conscious than others.
- ANewFormation 2y ago
  The fundamental problem is that we keep setting 'reasonable' benchmarks, implicitly because we want to make them seem reachable, but then when we inevitably do we find it feels we [of course] haven't achieved what we really wanted to in the first place. True artificial intelligence would have the ability to meaningfully and significantly improve itself, recursively - a characteristic of even the most primitive life on Earth. And the implications of a computing system capable of such simply can't be overstated. With access to essentially all contemporary knowledge, flawless recall, and stupidly ridiculous amounts of energy to power itself (relative to e.g. a brain), it would create an explosion of knowledge in every single domain imaginable, and many we can't even yet fathom. And it would be doing this at an accelerating pace owing to self improvement. Basically the very nature of existence and knowledge would change. Sentience or not would be irrelevant as we simply tried (and inevitably failed, miserably) to catch up. But this is a bit further out of reach than 'can vaguely pass for a human in a casual non-adversarial q&a style chat.' --- This was one of the rather many areas where Star Trek failed to really consider the implications of its concepts, probably because it would simply break the world building. The Borg, for instance, would be defacto Gods in no time, even without assimilation.
  - readyplayernull 2y ago
    > meaningfully and significantly improve itself Exactly, mechanically by tools, environmentally by science, metaphysically by philosophy. An AI can only become an I by crossing a singularity of knowledge.
  - captainclam 2y ago
    > This was one of the rather many areas where Star Trek failed to really consider the implications of its concepts, probably because it would simply break the world building. This is true of pretty much all scifi! It's funny seeing super-futuristic depictions of star-fighter pilots and combatants with firearms and its just...so crushingly evident that humans will not have supremacy in these arenas very shortly. Frank Herbert must have anticipated this complication and side-stepped the whole issue by preemptively canonizing the Butlerian Jihad.
jameshart 2y ago
Threepio reads so much like an LLM, the writing turns out to have been skockingly prescient. At the time of Phantom Menace, the whole story of him being assembled by some slave kid on a backwater planet in the outer rim seemed preposterous - why would a slave kid into pod racing make a pompous ass of a protocol droid? Well, he just installed a standard easily pirated LLM into the droid he made. And the fact that these LLMs are used as 'protocol droids' to communicate with lower-level devices - 'the binary language of moisture vaporators' etc. - matches up with the pattern of LLMs as tools for generating scripts in Python. Terrible though the last sequel is, they even managed to include jailbreaking Threepio to get him to access part of his training data that had been censored out.

nthingtohide 2y ago

Philosophers have coined the term P-zombies. LLMs can be considered as P-zombies if they do not possess consciousness. Functionally equivalent but no soul. Substance Dualism was used by philosophers to deny animals consciousness.

farts_mckensy 2y ago
Non materialist bullshit.
kbelder 2y ago
>Functionally equivalent but no soul. I think it's an incoherent concept. If the 'soul' is anything, there is some difference in the brain that would allow it. You can't have two people with identical brains and claim that one has a soul, unless you believe in magic.
- bee_rider 2y ago
  Is there a non-magical description of a soul? I thought it was an essentially magical concept. I’m not sure that makes it incoherent.
- bobxmax 2y ago
  It's been interesting seeing those with an entirely materialist understanding of the universe try to grapple with and deal with consciousness as LLMs have emerged. To maintain that belief you have to either believe there is a physical source of consciousness, or that consciousness doesn't exist, neither of which most people can accept, yet LLMs are forcing people to confront.
  - thfuran 2y ago
    You think most people can't accept that consciousness is a phenomenon with basis solely in physical reality?
    - Isamu 2y ago

superkuh 2y ago

Basically the plot of Peter Watts' "Blindsight" (post-2000) scifi book series.