5 ms·

Right? It's such a foreign form of intelligence to me. I think the paper "What is it like to be a bat" by Thomas Nagel made me realize that I can't even imagine

by sh-run 2y ago

Right? It's such a foreign form of intelligence to me. I think the paper "What is it like to be a bat" by Thomas Nagel made me realize that I can't even imagine what it's like to be my next door neighbor, let alone a being that has senses that differ from mine. Helen Keller's mind must work in a greatly different way than yours or mine. When I think, it's in English. I visualize things. Smell, touch and taste are never really involved. It's like they are the lesser of senses and yet that's all she had. It's incredible. Andy Weir in Project Hail Mary and Adrian Tchaikovsky in Children of [Time|Ruin|Memory] do a great job of describing what other forms of consciousness might be like, but still falls flat, I only really think in sight and sound. What is it like to be a bat? I'll never know.

evilduck 2y ago
Blindsight by Peter Watts also discusses what can be intelligent but not conscious. In the current hypefest of LLMs it’s interesting to consider that they may be similar.
- mikewarot 2y ago
  I think that LLMs might go through the reverse journey, being fluent in tokens (words-ish) and working backwards towards the physical reality we all inhabit.
- at_a_remove 2y ago
  "We do not like annoying cousins." Yes, exactly. The, uh, confident fluency of LLM responses, which can at the same time contradict what was said earlier, reminded me exactly of that. I don't know if you've ever met one of those glib psychopaths, but they have this characteristic of non-content communication, where it feels like words are being arranged for you, like someone composing a song using words from a language they do not know. See also: "you're talking a lot, but you're not saying anything."
  - ben_w 2y ago
    Hm. The contradictions specifically are a thing I notice in humans that I think are entirely normal[0]. But the early LLMs with the shorter context windows, those reminded me of my mum's Alzheimer's. That said, your analogy may well be perfect, as they are learning to people-please and to simulate things they (hopefully) don't actually experience. (Not that it changes your point, but isn't that Machiavellian rather than psychopathic?) [0] one of many reasons why I disagree with Wittgenstein about: > If there were a verb meaning 'to believe falsely', it would not have any significant first person, present indicative. Just because it's logically correct, doesn't mean humans think like that.
  - ohthehugemanate 2y ago
    The part that really gets ME about that thought, is that those glib psychopaths/sociopaths fill an important role in human society, generally as leaders. I'm sure we can all think of some prominent political figures who are very good at arranging words to get their audience excited, but have a tenuous connection to fact (at best). Actually factual content seems almost irrelevant to their ability to lead, or to their followers' desire to follow. If that's the function which we can now automate at scale, it's not the jobs the machines will ultimately take; it's the leadership.
- mannykannot 2y ago
  I was thinking the same. if there's anything that is what it is like to be an LLM (and I'm not saying that there is - in fact, I doubt it, while supposing that it is a possibility for future machines) I suspect it would be like this, but more so, and inverted: while Keller had some experience of an external world but no experience of language, the entire universe for an LLM is language, without any obvious way to suppose that this language is about an external world.
- qup 2y ago
  Available to read online, I read it last year: https://rifters.com/real/Blindsight.htm https://rifters.com/real/Blindsight.htm
- qbxk 2y ago
  i think the "problem" here is that for all of human history we have always been able to use mastery of language as a signal for intelligence and competence, of which LLMs are neither. it's possible this is even instinctual it's so ingrained in our concepts of "other minds". so we're going to have to get used to the fact that just using language well isn't enough to prove intelligence, certainly not consciousness. which then begs the the question, what is the magic ingredient, on top of use of language, that we have that bestows these qualities? and also the observation that whatever this ingredient is, it must be very difficult to measure or prove which is maybe why we stuck with the crude, but easy to wield, "use-of-language" test for so long.
antonioevans 2y ago
Children of Time/Ruin great two books. Highly recommend them if you like SciFi and animal behavior.
HeatrayEnjoyer 2y ago
I don't think it's that strange. My thoughts and my physical sensations are separate, imaging a different body different senses isn't that much of a stretch. I speak English but I don't think in it, thoughts don't have a language.
- blfr 2y ago
  I think that this is false, as in intersubjectively not true for the human experience. First, because our physical state has a huge influence on our thoughts, not just their content, but direction, "color." Secondly, and more importantly, while some thoughts may not have a language (image memories, mental maps), others certainly do, they're narrative. I only speak two languages but well enough (English is my second language) that I can think in both, and often come to a point where I have to decide which it will be for this train of thought. Shape rotators vs wordcels distinction strikes again, I guess.
  - frabcus 2y ago
    Quite a lot of people have no inner voice, others no inner imagery, others no inner unsymbolized conceptual thinking (cf all of Hurlburts research). We all use very varied modalities of thought! It's as rich as how different we look or how different we cook.
    - blfr 2y ago
      Having no inner voice, imagery, or whatever seems to be poorer rather than richer experience to me. I don't think the existence of deaf people invalidates the importance of music to human experience.
      - filleduchaos 2y ago
        I don't think a deaf person's inability to listen to music with their ears makes them incapable of depth and clarity of thought, no. I don't think people who aren't hard of hearing necessarily have particularly deep or clear thoughts simply because they listen to music with their ears either. It's very easy to confuse correlation with causation. (I've specified "with their ears" because deaf people can perceive music through other means than the cochlea + cochlear nerve.)
deleted 2y ago
[deleted]

by sh-run 2y ago

evilduck 2y ago

Blindsight by Peter Watts also discusses what can be intelligent but not conscious. In the current hypefest of LLMs it’s interesting to consider that they may be similar.

mikewarot 2y ago
I think that LLMs might go through the reverse journey, being fluent in tokens (words-ish) and working backwards towards the physical reality we all inhabit.
at_a_remove 2y ago
"We do not like annoying cousins." Yes, exactly. The, uh, confident fluency of LLM responses, which can at the same time contradict what was said earlier, reminded me exactly of that. I don't know if you've ever met one of those glib psychopaths, but they have this characteristic of non-content communication, where it feels like words are being arranged for you, like someone composing a song using words from a language they do not know. See also: "you're talking a lot, but you're not saying anything."
- ben_w 2y ago
  Hm. The contradictions specifically are a thing I notice in humans that I think are entirely normal[0]. But the early LLMs with the shorter context windows, those reminded me of my mum's Alzheimer's. That said, your analogy may well be perfect, as they are learning to people-please and to simulate things they (hopefully) don't actually experience. (Not that it changes your point, but isn't that Machiavellian rather than psychopathic?) [0] one of many reasons why I disagree with Wittgenstein about: > If there were a verb meaning 'to believe falsely', it would not have any significant first person, present indicative. Just because it's logically correct, doesn't mean humans think like that.
- ohthehugemanate 2y ago
  The part that really gets ME about that thought, is that those glib psychopaths/sociopaths fill an important role in human society, generally as leaders. I'm sure we can all think of some prominent political figures who are very good at arranging words to get their audience excited, but have a tenuous connection to fact (at best). Actually factual content seems almost irrelevant to their ability to lead, or to their followers' desire to follow. If that's the function which we can now automate at scale, it's not the jobs the machines will ultimately take; it's the leadership.
mannykannot 2y ago
I was thinking the same. if there's anything that is what it is like to be an LLM (and I'm not saying that there is - in fact, I doubt it, while supposing that it is a possibility for future machines) I suspect it would be like this, but more so, and inverted: while Keller had some experience of an external world but no experience of language, the entire universe for an LLM is language, without any obvious way to suppose that this language is about an external world.
qup 2y ago
Available to read online, I read it last year: https://rifters.com/real/Blindsight.htm https://rifters.com/real/Blindsight.htm
qbxk 2y ago
i think the "problem" here is that for all of human history we have always been able to use mastery of language as a signal for intelligence and competence, of which LLMs are neither. it's possible this is even instinctual it's so ingrained in our concepts of "other minds". so we're going to have to get used to the fact that just using language well isn't enough to prove intelligence, certainly not consciousness. which then begs the the question, what is the magic ingredient, on top of use of language, that we have that bestows these qualities? and also the observation that whatever this ingredient is, it must be very difficult to measure or prove which is maybe why we stuck with the crude, but easy to wield, "use-of-language" test for so long.

antonioevans 2y ago

Children of Time/Ruin great two books. Highly recommend them if you like SciFi and animal behavior.

HeatrayEnjoyer 2y ago

I don't think it's that strange. My thoughts and my physical sensations are separate, imaging a different body different senses isn't that much of a stretch. I speak English but I don't think in it, thoughts don't have a language.

blfr 2y ago
I think that this is false, as in intersubjectively not true for the human experience. First, because our physical state has a huge influence on our thoughts, not just their content, but direction, "color." Secondly, and more importantly, while some thoughts may not have a language (image memories, mental maps), others certainly do, they're narrative. I only speak two languages but well enough (English is my second language) that I can think in both, and often come to a point where I have to decide which it will be for this train of thought. Shape rotators vs wordcels distinction strikes again, I guess.
- frabcus 2y ago
  Quite a lot of people have no inner voice, others no inner imagery, others no inner unsymbolized conceptual thinking (cf all of Hurlburts research). We all use very varied modalities of thought! It's as rich as how different we look or how different we cook.
  - blfr 2y ago
    Having no inner voice, imagery, or whatever seems to be poorer rather than richer experience to me. I don't think the existence of deaf people invalidates the importance of music to human experience.
    - filleduchaos 2y ago
      I don't think a deaf person's inability to listen to music with their ears makes them incapable of depth and clarity of thought, no. I don't think people who aren't hard of hearing necessarily have particularly deep or clear thoughts simply because they listen to music with their ears either. It's very easy to confuse correlation with causation. (I've specified "with their ears" because deaf people can perceive music through other means than the cochlea + cochlear nerve.)

deleted 2y ago

[deleted]