9 ms·

Ok I like your thought experiment. Lets change it a bit. Instead of it being an unknown language, its English (a language you know), but every single Noun, Ver

Ok I like your thought experiment. Lets change it a bit. Instead of it being an unknown language, its English (a language you know), but every single Noun, Verb, Adjective or Preposition has been changed to Thai (a language you dont know). The Mæw Nạ̀ng Bn the S̄eụ̄̀x. If you had sufficient opportunity to study this pile of text, you'd begin to pick out patterns of which words appear together, and what order words often appear in. Suppose you study this giant stack of Thai text for years in isolation. After all this study, you're good enough that given a few written Thai words, you can write sequences of words that are likely to follow, given what you know of these patterns. Right, and to get good at this task, you'd need to build models in your head. You would think to yourself, right a Mæw tends to nạ̀ng bn a S̄eụ̄̀x, and you would build up a model of the sort of things a Mæw might do, the situations it might be in. In an abstract way. As you absorbed more and more data you would adjust these abstract models to fit the evidence you had. You dont know what a Mæw is. But if someone asks you about a Mæw, you can talk about how it relates to S̄eụ̄̀x, Plā and H̄nū. You know stuff about Mæw, but its abstract.

hzay 4y ago
I love this theory. You're saying that a distinction can be drawn between our linguistic concepts and our lived experience, and that the former can be learned without the latter. And that a model could operate upon those linguistic concepts in a useful way, but without the benefit (or drawback?) of the mappings we keep between language and experience. And that it can learn this based on the large amount of texts we have. Fascinating, and seems like a plausible description of what's going on.
schrodinger 4y ago
Basically the "rosetta stone" theory, right?
abeppu 4y ago
If you're constructing this to rely on my prior knowledge both of the world and of English, then I must remind you that those are things the LLM does not have. We have to be careful to not allow our human inferential biases from distorting our thinking about that the models are doing.
- didntreadarticl 4y ago
  I know that. its a metaphor to adjust the 'Thai Language' intuition-pump that was presented. I'm making it easier to imagine how a Large Language Model might make a Model of the Language
- polishdude20 4y ago
  Yeah but if you ask the model what a cat is, it'll use other words that describe a cat because they're usually used in a sentence about cats. These words must relate to cats. So if I ask you what a cat is, you'll use words that relate to cats. Sure, you may visually see these words in your head. You may visually see a cat in your head, but your output to me is just a description of a cat. That's the same thing the network would do.
  - abeppu 4y ago
    The whole point of this conversation is whether talking like an agent that has a theory of mind and actually having a theory of mind are the same thing. I responded to a thread about what "knowing" is, and the same distinction can apply. You're responding with "if it talks like it knows what a cat is, it must know what a cat is", and that's totally begging the question.
    - d1sxeyes 4y ago
      While I agree with your point, how would you test that? How could you determine whether an LLM “knows” what a cat is. And what is “knowing”? If I know that a Mæw tends to nạ̀ng bn a S̄eụ̄̀x, isn’t that the first thing I’ve learned? And couldn’t I continue to learn other properties of Mæws? How many do I need to learn to “know” what a Mæw is?
      - pegasus 4y ago
        Like GP said, the LLM has no chance at knowing what a cat is, regardless of how much data it ingests, because a cat is not made of data. It's not like you're getting closer and closer to knowing what a "Mæw" is. You were at the same remote distance all the time. This is called the "grounding problem" in AI. As for how you would test it, I think one-shot learning would get one closer to proving understanding.

hzay 4y ago

I love this theory. You're saying that a distinction can be drawn between our linguistic concepts and our lived experience, and that the former can be learned without the latter. And that a model could operate upon those linguistic concepts in a useful way, but without the benefit (or drawback?) of the mappings we keep between language and experience. And that it can learn this based on the large amount of texts we have. Fascinating, and seems like a plausible description of what's going on.

schrodinger 4y ago

Basically the "rosetta stone" theory, right?

abeppu 4y ago

If you're constructing this to rely on my prior knowledge both of the world and of English, then I must remind you that those are things the LLM does not have. We have to be careful to not allow our human inferential biases from distorting our thinking about that the models are doing.

didntreadarticl 4y ago
I know that. its a metaphor to adjust the 'Thai Language' intuition-pump that was presented. I'm making it easier to imagine how a Large Language Model might make a Model of the Language
polishdude20 4y ago
Yeah but if you ask the model what a cat is, it'll use other words that describe a cat because they're usually used in a sentence about cats. These words must relate to cats. So if I ask you what a cat is, you'll use words that relate to cats. Sure, you may visually see these words in your head. You may visually see a cat in your head, but your output to me is just a description of a cat. That's the same thing the network would do.
- abeppu 4y ago
  The whole point of this conversation is whether talking like an agent that has a theory of mind and actually having a theory of mind are the same thing. I responded to a thread about what "knowing" is, and the same distinction can apply. You're responding with "if it talks like it knows what a cat is, it must know what a cat is", and that's totally begging the question.
  - d1sxeyes 4y ago
    While I agree with your point, how would you test that? How could you determine whether an LLM “knows” what a cat is. And what is “knowing”? If I know that a Mæw tends to nạ̀ng bn a S̄eụ̄̀x, isn’t that the first thing I’ve learned? And couldn’t I continue to learn other properties of Mæws? How many do I need to learn to “know” what a Mæw is?
    - pegasus 4y ago
      Like GP said, the LLM has no chance at knowing what a cat is, regardless of how much data it ingests, because a cat is not made of data. It's not like you're getting closer and closer to knowing what a "Mæw" is. You were at the same remote distance all the time. This is called the "grounding problem" in AI. As for how you would test it, I think one-shot learning would get one closer to proving understanding.