14 ms·

This highlights one of the types of muddled thinking around LLMs. These tasks are used to test theory of mind because for people, language is a reliable represe

by lsy 4y ago

This highlights one of the types of muddled thinking around LLMs. These tasks are used to test theory of mind because for people, language is a reliable representation of what type of thoughts are going on in the person's mind. In the case of an LLM the language generated doesn't have the same relationship to reality as it does for a person. What is being demonstrated in the article is that given billions of tokens of human-written training data, a statistical model can generate text that satisfies some of our expectations of how a person would respond to this task. Essentially we have enough parameters to capture from existing writing that statistically, the most likely word following "she looked in the bag labelled (X), and saw that it was full of (NOT X). She felt " is "surprised" or "confused" or some other word that is commonly embedded alongside contradictions. What this article is not showing (but either irresponsibly or naively suggests) is that the LLM knows what a bag is, what a person is, what popcorn and chocolate are, and can then put itself in the shoes of someone experiencing this situation, and finally communicate its own theory of what is going on in that person's mind. That is just not in evidence. The discussion is also muddled, saying that if structural properties of language create the ability to solve these tasks, then the tasks are either useless for studying humans, or suggest that humans can solve these tasks without ToM. The alternative explanation is of course that humans are known to be not-great at statistical next-word guesses (see Family Feud for examples), but are also known to use language to accurately describe their internal mental states. So the tasks remain useful and accurate in testing ToM in people because people can't perform statistical regressions over billion-token sets and therefore must generate their thoughts the old fashioned way.

naasking 4y ago
> What this article is not showing (but either irresponsibly or naively suggests) is that the LLM knows what a bag is, what a person is, what popcorn and chocolate are, and can then put itself in the shoes of someone experiencing this situation, and finally communicate its own theory of what is going on in that person's mind. That is just not in evidence. You can't really conclude that unless you think we have a deep mechanistic understanding of "knowing". I agree that LLM doesn't have the same knowledge of these things as a human does, but it clearly has some kind of knowledge of how these words relate to each other. It "knows" that a "person" "puts" "things" "in" "bags", and for instance, that bags don't put things in people. So it clearly has some knowledge of bags and people, it just doesn't have multisensory associations with these objects.
Jipazgqmnm 4y ago
It's called Chinese Room: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room > The question Searle wants to answer is this: does the machine literally "understand" Chinese? Or is it merely simulating the ability to understand Chinese? To me: If you can't tell, it effectively doesn't matter.
acuozzo 4y ago
> language is a reliable representation of what type of thoughts are going on in the person's mind The wording used here inherently rejects Linguistic Determinism and, to a lesser extent, Linguistic Relativism.
unkulunkulu 4y ago
Today at the zoo I saw chimpanzees and right next to their area there was a fun fact tablet. It said that before around 1960 it was thought that humans were the only species to use tools. After discovering the same for chimps, Louis Leaky said “Now we must redefine tool, redefine man, or accept chimpanzees as human.”
spuz 4y ago
> So the tasks remain useful and accurate in testing ToM in people because people can't perform statistical regressions over billion-token sets and therefore must generate their thoughts the old fashioned way. Is it not also possible that the study suggests that the human mind actually operates as a statistical regression over billions of data points rather than through some kind of Baysian logic? You say humans are known to be not-great at statistical next-word guesses, but I would antelope they're actually pretty good at it.
- pdonis 4y ago
  > Is it not also possible that the study suggests that the human mind actually operates as a statistical regression over billions of data points rather than through some kind of Baysian logic? No. Human minds have semantic relationships to the rest of the world that LLMs do not have. The comparisons being made between the two, not just in this paper but in all of the hype surrounding LLMs, are simply invalid. But they sure help in collecting more funding.
  - mietek 4y ago
    Please define “semantic relationship”.
    - pdonis 4y ago
      See my response to strbean upthread.
      - mietek 4y ago
        Okay. It seems to me that, in analogy to humans having such a relationship with physical reality, machine learning systems have such a relationship with virtual reality — the environment in which they are trained and deployed.
        pdonis 4y ago
        > machine learning systems have such a relationship with virtual reality No, they don't, because they can't take actions in that virtual reality and sense the consequences. They can't test hypotheses about how the reality works. They can't even frame hypotheses about how the reality works.
didntreadarticl 4y ago
That is just not in evidence. Seems like its nailing it to me. You ask about a scenario and it gives an appropriate answer. We have evidence that LLMs build models of the things they are learning about. Have a look at this paper: Do Large Language Models learn world models or just surface statistics? https://thegradient.pub/othello/ https://thegradient.pub/othello/ previously discussed https://news.ycombinator.com/item?id=34474043 https://news.ycombinator.com/item?id=34474043
pdonis 4y ago
> In the case of an LLM the language generated doesn't have the same relationship to reality as it does for a person. I wish I could upvote this 1000 times. It is the core issue that all the hype surrounding LLMs consistently fails to address or even acknowledge.
- didntreadarticl 4y ago
  OK how about this: It 'knows' language, as in it has learnt about relationships between words (and thats really underselling it, in reality it has learnt very very subtle relationships between a great many words, and it can process words about 2000 at a time (token count etc)) BUT as you say it has no outside reference, its just a bundle of weights (those weights forming models of a sort) BUT we provide the outside context by interacting with it. We ask it a question, it is able to provide an answer. In any case it wont be long before someone hooks one of these up to cameras and robot arms and teaches it to make a cup of tea or whatever. A 'relationship to reality' is coming in the next few years if you think thats a critical ingredient.
  - swatcoder 4y ago
    The future AI you describe has language at the center of its understanding. For example, you would expect that the cameras and arm-feedback-sensors would produce text (or text-associated weights/tokens) that describe what the AI "sees" and the robotic arms would be receive some kind of language-derived directives from the LLM. It will be very interesting to see what that system is capable of. I think a lot of people here don't identify language as an essential part of "thinking" and "being", and instead view language as a superficial layer whose role is primarily about social communication and secondarily about abstract reasoning. This is why some of us see these LLM examples as not really speaking to intelligence per se. It's hard for some of us to conceive of intelligence as being represented purely in language. So your proposed system would be a extremely interesting exploration of that! I look forward to it.
    - didntreadarticl 4y ago
      So your proposed system would be a extremely interesting exploration of that! I look forward to it. Right, so lets virtualise it. Actually training AIs using real cameras and real robot arms will be really slow and expensive. So we provide a system that renders a photorealistic graphical room with teapot and robot arm, and a virtual camera inside the room is 'seeing' parts of the room and a vision model then processes what it can 'see' to try and feed info to the LLM. Likewise the LLM can make the robot arm move but its all just simulated. Does the LLM now have a relationship with reality?
canniballectern 4y ago
Of course, today's LLMs only appear to have theory of mind at first glance and fall apart under closer scrutiny. But if they can continue to become more and more accurate replicas of the real thing, I don't think it matters at all. There's no way to know for sure that anyone other than yourself experiences consciousness. All you can do is judge for yourself that what they're describing matches closely enough with your own experiences that they're probably experiencing the same thing you are.
- robertlagrant 4y ago
  > But if they can continue to become more and more accurate replicas of the real thing, I don't think it matters at all. So, I suppose I'd ask: what does "matter" mean here? If you knew that everyone you loved had been destroyed and been replaced by exact replicas, would that matter?
  - sgt101 4y ago
    Star trek universe!
  - canniballectern 4y ago
    If the replicas were truly exact, I guess not ¯\_(ツ)_/¯
  - bigbluedots 4y ago
    That already happens frequently enough at a cellular level.
- pdonis 4y ago
  > There's no way to know for sure that anyone other than yourself experiences consciousness. All you can do is judge for yourself that what they're describing matches closely enough with your own experiences that they're probably experiencing the same thing you are. That judgment is not just based on the words other people use. It is based on knowing that other people's brains and minds have the same sort of semantic relationships to the rest of the world that yours do. And those relationships can be tested by checking to see if, for example, the other person uses the same words to refer to particular objects in the real world that you do, or if they react to particular real-world events in the same way that you do. You can't even test any of this with an LLM because the LLM simply does not have the same kind of semantic relationships with the rest of the world that you do. It has no such relationships at all.
kenjackson 4y ago
It seems obvious that LLMs don’t understand a bag the way we do. It’s never seen a bag. Or held one. But if you equip it with the same IO as humans, how different would it be? Probably still pretty different, but light years closer than what we had 20 years ago. Also humans are good at next word guessing their own next word. Each person has been trained on a different set of data, so it’s no surprise that they wouldn’t be able to guess other people’s next words.
space_fountain 4y ago
I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of kind rather than degree. Some interesting facts that point to it being a difference of degree. LLM are actually are more accurate when asked to explain their thinking. They make similar mistakes to humans intuitive reasoning. It might help to define what we even mean by knowing things. To me being able to make novel predictions that require the knowledge is the only definition one could use that doesn’t run into the possibility of deciding humans don’t actually know anything
- ves 4y ago
  > I find that especially important because to every appearance we are a machine learning algorithm. Speak for yourself.
- robertlagrant 4y ago
  > It might help to define what we even mean by knowing things This is it, I think. It's interesting that we now have a practical example to point at when asking formerly-abstruse philosophical questions.
- colechristensen 4y ago
  The kinds of mistakes something makes are a strong indicator of whether answers are a product of understanding or memorization.
- avgcorrection 4y ago
  The problem with this facile view of things is that it seems to be a dead end for scientific theories. What if we just limited the science of birds to explaining how limb-flapping could produce levitation? Hmm yes. Birds are kind of like helicopters, it seems. Who’s to say that they are not basically one and the same? Moving on. If you are only interested in the most superficial tests and theories—like the Turing Test—then consider psychology conquered once you’ve tricked a human with your chat bot. Game Over. And what did you learn...?
low_tech_punk 4y ago
> What this article is not showing (but either irresponsibly or naively suggests) is that the LLM knows what a bag is, what a person is, what popcorn and chocolate are, and can then put itself in the shoes of someone experiencing this situation... Does a human truly know? Feels like a slippery slope to the qualia question where we can't agree on what it means for the human to feel a human experience.
didntreadarticl 4y ago
LLMs cant think and submarines cant swim
avgcorrection 4y ago
The abstract sounds about as significant as when “AI”, which is both championed and feared by people who superstitiously imagine dystopian run-amock AI intent on killing off humanity, itself expresses that very same idea. Really? Your trained program comes up with derivative stories told by computer nerds?
draw_down 4y ago
[dead]
tialaramex 4y ago
You don't seem to have understood what was tested? The model answered the keyword prompt and spontaneously offered more details. That is, the authors were interested in whether it says "Popcorn" or "Chocolate" (or something else entirely) when the correct answer is "Popcorn" and not only does GPT-3 almost always choose "Popcorn" it also follows on to justify that by explaining that the subject is surprised. The full data set isn't available yet (the author said they intend to provide it on the 9th of February, I suppose it's possible they'll get to it this evening) but one of the most interesting things would be what are the weirder answers. If a model says "Popcorn" 98% of the time, and "Chocolate" 0% of the time, that leaves 2% weird answers. Maybe it sometimes says "Popped corn" or "Sweet treat" or something reasonable but maybe it's fully crazy, if you talk about a bag of Popcorn labelled as Chocolate but the model sometimes picks "A fire-breathing lizard" that's pretty weird right ?
mhitza 4y ago
> These tasks are used to test theory of mind because for people, language is a reliable representation of what type of thoughts are going on in the person's mind. I actually think that human language is unreliable at expressing what's going on inside a persons mind[1]. My native language is not English, I have only introductory-level knowledge in the field of pragmatics[2], which makes me fully aware of the many ways in which I could fail to write a compelling sentence to support my argument. I can use language to only approximate thoughts in my head, and when it comes to abstract concepts and ideas, words alone, I assert, are never sufficient. It isn't even necessary to step outside our main knowledge area to illustrate this point. How many around here have read a Monad tutorial, without any hands-on experience, and how many of those have understood what Monads are/or how they work from words alone? My entire paragraph from before, just to set the stage on a simple question. How can you even formulate a question, for a multi-billion parameter language model, to evaluate that it can understand in an abstract/conceptual way something/anything? Heck, how can you do that with other people? I think if we'd have an answer here, we actually could evaluate easily experience/expertise with anyone we'd interview; instead of requiring credentials, references, tests, trials, etc. [1] Lots of poets from the romanticism era liked to touch upon this topic. One that comes to mind, and one of my personal favorites, is Silentium by Fyodor Tyutchev https://culturedarm.com/silentium-by-fyodor-tyutchev/ https://culturedarm.com/silentium-by-fyodor-tyutchev/ [2] https://en.wikipedia.org/wiki/Pragmatics https://en.wikipedia.org/wiki/Pragmatics
- HTTP418 4y ago
  This reminds me of the Trisolaran from the three-body problem. <spoiler> They communicate with each other much more accurately and quickly with each other and have no need for a speech/language per se </spoiler>
albystein 4y ago
I think this line of reasoning is misguided. What’s striking and more important to focus on are the abstract reasoning abilities of these systems. Language, as you mentioned, abstracts real world objects and phenomena, so it’s a good approximation of the real world. Thus, if an LLM can reason this well using language, it’s safe to say that perhaps they’re doing something akin to what the human mind does. Your critique about lack of grounding in these systems is an easy problem to solve. It’s as easy as teaching an LLM to associate words with real world objects or phenomena. Image-classification models, text-2-image models, audio transcription models, and many other modal specific systems already do this to some extent. And more recently there has been a push towards multi-modal language models(Deepmind’s flamingo), so this line of argument will be debunked very soon. I actually believe GPT-4 will be multi-modal and it’s capabilities will dispel majority of these criticisms
- lee101 4y ago
  [flagged]
- Syntonicles 4y ago
  I agree. We've [potentially] given the grounding assimilation step a boost already, since language is already organized. Imagine that a language model is fully integrated with sense-data that exceeds human first-hand experience. Perhaps they are trained on and can generate realistic 3D models of objects, and derive estimates of their internal construction, weight, etc. Perhaps they recall infrared emissions or opacity to EM wavelengths. Would we truly "know" what we're talking about by that standard? I'm not actually sure why we don't consider generative image models to be grounded already. They seem to be able to modify, transform and rotate imagery. That indicates spatial understanding to me, and I'm not sure how much more we must require of them without having to exclude blind or otherwise disabled humans from our definition of comprehension.
nl 4y ago
This objection is the well known Chinese Room Experiment objection[1]. The issue is that there doesn't seem to be a better alternative. Either we build intelligence tests that some variety of the Chinese Room experiment will pass, or * We have to consider that humans aren't intelligent by our own definitions (or rarely so). * We decide intelligence isn't actually a scientific attribute and is more akin to a religious attribute, so we abandon the idea of being able to test if something is intelligent. [1] https://plato.stanford.edu/entries/chinese-room/ https://plato.stanford.edu/entries/chinese-room/
onlyrealcuzzo 4y ago
How do I know my thoughts aren't statistical noise?
- qbrass 4y ago
  Keep telling yourself they aren't. Eventually you'll know it's true.

by lsy 4y ago

naasking 4y ago

> What this article is not showing (but either irresponsibly or naively suggests) is that the LLM knows what a bag is, what a person is, what popcorn and chocolate are, and can then put itself in the shoes of someone experiencing this situation, and finally communicate its own theory of what is going on in that person's mind. That is just not in evidence. You can't really conclude that unless you think we have a deep mechanistic understanding of "knowing". I agree that LLM doesn't have the same knowledge of these things as a human does, but it clearly has some kind of knowledge of how these words relate to each other. It "knows" that a "person" "puts" "things" "in" "bags", and for instance, that bags don't put things in people. So it clearly has some knowledge of bags and people, it just doesn't have multisensory associations with these objects.

Jipazgqmnm 4y ago

It's called Chinese Room: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room > The question Searle wants to answer is this: does the machine literally "understand" Chinese? Or is it merely simulating the ability to understand Chinese? To me: If you can't tell, it effectively doesn't matter.

acuozzo 4y ago

> language is a reliable representation of what type of thoughts are going on in the person's mind The wording used here inherently rejects Linguistic Determinism and, to a lesser extent, Linguistic Relativism.

unkulunkulu 4y ago

Today at the zoo I saw chimpanzees and right next to their area there was a fun fact tablet. It said that before around 1960 it was thought that humans were the only species to use tools. After discovering the same for chimps, Louis Leaky said “Now we must redefine tool, redefine man, or accept chimpanzees as human.”

spuz 4y ago

> So the tasks remain useful and accurate in testing ToM in people because people can't perform statistical regressions over billion-token sets and therefore must generate their thoughts the old fashioned way. Is it not also possible that the study suggests that the human mind actually operates as a statistical regression over billions of data points rather than through some kind of Baysian logic? You say humans are known to be not-great at statistical next-word guesses, but I would antelope they're actually pretty good at it.

pdonis 4y ago
> Is it not also possible that the study suggests that the human mind actually operates as a statistical regression over billions of data points rather than through some kind of Baysian logic? No. Human minds have semantic relationships to the rest of the world that LLMs do not have. The comparisons being made between the two, not just in this paper but in all of the hype surrounding LLMs, are simply invalid. But they sure help in collecting more funding.
- mietek 4y ago
  Please define “semantic relationship”.
  - pdonis 4y ago
    See my response to strbean upthread.
    - mietek 4y ago
      Okay. It seems to me that, in analogy to humans having such a relationship with physical reality, machine learning systems have such a relationship with virtual reality — the environment in which they are trained and deployed.
      - pdonis 4y ago
        > machine learning systems have such a relationship with virtual reality No, they don't, because they can't take actions in that virtual reality and sense the consequences. They can't test hypotheses about how the reality works. They can't even frame hypotheses about how the reality works.

didntreadarticl 4y ago

That is just not in evidence. Seems like its nailing it to me. You ask about a scenario and it gives an appropriate answer. We have evidence that LLMs build models of the things they are learning about. Have a look at this paper: Do Large Language Models learn world models or just surface statistics? https://thegradient.pub/othello/ https://thegradient.pub/othello/ previously discussed https://news.ycombinator.com/item?id=34474043 https://news.ycombinator.com/item?id=34474043

pdonis 4y ago

> In the case of an LLM the language generated doesn't have the same relationship to reality as it does for a person. I wish I could upvote this 1000 times. It is the core issue that all the hype surrounding LLMs consistently fails to address or even acknowledge.

didntreadarticl 4y ago
OK how about this: It 'knows' language, as in it has learnt about relationships between words (and thats really underselling it, in reality it has learnt very very subtle relationships between a great many words, and it can process words about 2000 at a time (token count etc)) BUT as you say it has no outside reference, its just a bundle of weights (those weights forming models of a sort) BUT we provide the outside context by interacting with it. We ask it a question, it is able to provide an answer. In any case it wont be long before someone hooks one of these up to cameras and robot arms and teaches it to make a cup of tea or whatever. A 'relationship to reality' is coming in the next few years if you think thats a critical ingredient.
- swatcoder 4y ago
  The future AI you describe has language at the center of its understanding. For example, you would expect that the cameras and arm-feedback-sensors would produce text (or text-associated weights/tokens) that describe what the AI "sees" and the robotic arms would be receive some kind of language-derived directives from the LLM. It will be very interesting to see what that system is capable of. I think a lot of people here don't identify language as an essential part of "thinking" and "being", and instead view language as a superficial layer whose role is primarily about social communication and secondarily about abstract reasoning. This is why some of us see these LLM examples as not really speaking to intelligence per se. It's hard for some of us to conceive of intelligence as being represented purely in language. So your proposed system would be a extremely interesting exploration of that! I look forward to it.
  - didntreadarticl 4y ago
    So your proposed system would be a extremely interesting exploration of that! I look forward to it. Right, so lets virtualise it. Actually training AIs using real cameras and real robot arms will be really slow and expensive. So we provide a system that renders a photorealistic graphical room with teapot and robot arm, and a virtual camera inside the room is 'seeing' parts of the room and a vision model then processes what it can 'see' to try and feed info to the LLM. Likewise the LLM can make the robot arm move but its all just simulated. Does the LLM now have a relationship with reality?

canniballectern 4y ago

Of course, today's LLMs only appear to have theory of mind at first glance and fall apart under closer scrutiny. But if they can continue to become more and more accurate replicas of the real thing, I don't think it matters at all. There's no way to know for sure that anyone other than yourself experiences consciousness. All you can do is judge for yourself that what they're describing matches closely enough with your own experiences that they're probably experiencing the same thing you are.

robertlagrant 4y ago
> But if they can continue to become more and more accurate replicas of the real thing, I don't think it matters at all. So, I suppose I'd ask: what does "matter" mean here? If you knew that everyone you loved had been destroyed and been replaced by exact replicas, would that matter?
- sgt101 4y ago
  Star trek universe!
- canniballectern 4y ago
  If the replicas were truly exact, I guess not ¯\_(ツ)_/¯
- bigbluedots 4y ago
  That already happens frequently enough at a cellular level.
pdonis 4y ago
> There's no way to know for sure that anyone other than yourself experiences consciousness. All you can do is judge for yourself that what they're describing matches closely enough with your own experiences that they're probably experiencing the same thing you are. That judgment is not just based on the words other people use. It is based on knowing that other people's brains and minds have the same sort of semantic relationships to the rest of the world that yours do. And those relationships can be tested by checking to see if, for example, the other person uses the same words to refer to particular objects in the real world that you do, or if they react to particular real-world events in the same way that you do. You can't even test any of this with an LLM because the LLM simply does not have the same kind of semantic relationships with the rest of the world that you do. It has no such relationships at all.

kenjackson 4y ago

It seems obvious that LLMs don’t understand a bag the way we do. It’s never seen a bag. Or held one. But if you equip it with the same IO as humans, how different would it be? Probably still pretty different, but light years closer than what we had 20 years ago. Also humans are good at next word guessing their own next word. Each person has been trained on a different set of data, so it’s no surprise that they wouldn’t be able to guess other people’s next words.

space_fountain 4y ago

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of kind rather than degree. Some interesting facts that point to it being a difference of degree. LLM are actually are more accurate when asked to explain their thinking. They make similar mistakes to humans intuitive reasoning. It might help to define what we even mean by knowing things. To me being able to make novel predictions that require the knowledge is the only definition one could use that doesn’t run into the possibility of deciding humans don’t actually know anything

ves 4y ago
> I find that especially important because to every appearance we are a machine learning algorithm. Speak for yourself.
robertlagrant 4y ago
> It might help to define what we even mean by knowing things This is it, I think. It's interesting that we now have a practical example to point at when asking formerly-abstruse philosophical questions.
colechristensen 4y ago
The kinds of mistakes something makes are a strong indicator of whether answers are a product of understanding or memorization.
avgcorrection 4y ago
The problem with this facile view of things is that it seems to be a dead end for scientific theories. What if we just limited the science of birds to explaining how limb-flapping could produce levitation? Hmm yes. Birds are kind of like helicopters, it seems. Who’s to say that they are not basically one and the same? Moving on. If you are only interested in the most superficial tests and theories—like the Turing Test—then consider psychology conquered once you’ve tricked a human with your chat bot. Game Over. And what did you learn...?

low_tech_punk 4y ago

> What this article is not showing (but either irresponsibly or naively suggests) is that the LLM knows what a bag is, what a person is, what popcorn and chocolate are, and can then put itself in the shoes of someone experiencing this situation... Does a human truly know? Feels like a slippery slope to the qualia question where we can't agree on what it means for the human to feel a human experience.

didntreadarticl 4y ago

LLMs cant think and submarines cant swim

avgcorrection 4y ago

The abstract sounds about as significant as when “AI”, which is both championed and feared by people who superstitiously imagine dystopian run-amock AI intent on killing off humanity, itself expresses that very same idea. Really? Your trained program comes up with derivative stories told by computer nerds?

draw_down 4y ago

[dead]

tialaramex 4y ago

You don't seem to have understood what was tested? The model answered the keyword prompt and spontaneously offered more details. That is, the authors were interested in whether it says "Popcorn" or "Chocolate" (or something else entirely) when the correct answer is "Popcorn" and not only does GPT-3 almost always choose "Popcorn" it also follows on to justify that by explaining that the subject is surprised. The full data set isn't available yet (the author said they intend to provide it on the 9th of February, I suppose it's possible they'll get to it this evening) but one of the most interesting things would be what are the weirder answers. If a model says "Popcorn" 98% of the time, and "Chocolate" 0% of the time, that leaves 2% weird answers. Maybe it sometimes says "Popped corn" or "Sweet treat" or something reasonable but maybe it's fully crazy, if you talk about a bag of Popcorn labelled as Chocolate but the model sometimes picks "A fire-breathing lizard" that's pretty weird right ?

mhitza 4y ago

> These tasks are used to test theory of mind because for people, language is a reliable representation of what type of thoughts are going on in the person's mind. I actually think that human language is unreliable at expressing what's going on inside a persons mind[1]. My native language is not English, I have only introductory-level knowledge in the field of pragmatics[2], which makes me fully aware of the many ways in which I could fail to write a compelling sentence to support my argument. I can use language to only approximate thoughts in my head, and when it comes to abstract concepts and ideas, words alone, I assert, are never sufficient. It isn't even necessary to step outside our main knowledge area to illustrate this point. How many around here have read a Monad tutorial, without any hands-on experience, and how many of those have understood what Monads are/or how they work from words alone? My entire paragraph from before, just to set the stage on a simple question. How can you even formulate a question, for a multi-billion parameter language model, to evaluate that it can understand in an abstract/conceptual way something/anything? Heck, how can you do that with other people? I think if we'd have an answer here, we actually could evaluate easily experience/expertise with anyone we'd interview; instead of requiring credentials, references, tests, trials, etc. [1] Lots of poets from the romanticism era liked to touch upon this topic. One that comes to mind, and one of my personal favorites, is Silentium by Fyodor Tyutchev https://culturedarm.com/silentium-by-fyodor-tyutchev/ https://culturedarm.com/silentium-by-fyodor-tyutchev/ [2] https://en.wikipedia.org/wiki/Pragmatics https://en.wikipedia.org/wiki/Pragmatics

HTTP418 4y ago
This reminds me of the Trisolaran from the three-body problem. <spoiler> They communicate with each other much more accurately and quickly with each other and have no need for a speech/language per se </spoiler>

albystein 4y ago

I think this line of reasoning is misguided. What’s striking and more important to focus on are the abstract reasoning abilities of these systems. Language, as you mentioned, abstracts real world objects and phenomena, so it’s a good approximation of the real world. Thus, if an LLM can reason this well using language, it’s safe to say that perhaps they’re doing something akin to what the human mind does. Your critique about lack of grounding in these systems is an easy problem to solve. It’s as easy as teaching an LLM to associate words with real world objects or phenomena. Image-classification models, text-2-image models, audio transcription models, and many other modal specific systems already do this to some extent. And more recently there has been a push towards multi-modal language models(Deepmind’s flamingo), so this line of argument will be debunked very soon. I actually believe GPT-4 will be multi-modal and it’s capabilities will dispel majority of these criticisms

lee101 4y ago
[flagged]
Syntonicles 4y ago
I agree. We've [potentially] given the grounding assimilation step a boost already, since language is already organized. Imagine that a language model is fully integrated with sense-data that exceeds human first-hand experience. Perhaps they are trained on and can generate realistic 3D models of objects, and derive estimates of their internal construction, weight, etc. Perhaps they recall infrared emissions or opacity to EM wavelengths. Would we truly "know" what we're talking about by that standard? I'm not actually sure why we don't consider generative image models to be grounded already. They seem to be able to modify, transform and rotate imagery. That indicates spatial understanding to me, and I'm not sure how much more we must require of them without having to exclude blind or otherwise disabled humans from our definition of comprehension.

nl 4y ago

This objection is the well known Chinese Room Experiment objection[1]. The issue is that there doesn't seem to be a better alternative. Either we build intelligence tests that some variety of the Chinese Room experiment will pass, or * We have to consider that humans aren't intelligent by our own definitions (or rarely so). * We decide intelligence isn't actually a scientific attribute and is more akin to a religious attribute, so we abandon the idea of being able to test if something is intelligent. [1] https://plato.stanford.edu/entries/chinese-room/ https://plato.stanford.edu/entries/chinese-room/

onlyrealcuzzo 4y ago

How do I know my thoughts aren't statistical noise?

qbrass 4y ago
Keep telling yourself they aren't. Eventually you'll know it's true.