4 ms·

I would argue they do not contain knowledge using the classic definition of knowledge as "justified true belief." They can emit statements that happen to be tr

by lukev 3y ago

I would argue they do not contain knowledge using the classic definition of knowledge as "justified true belief." They can emit statements that happen to be true because the probabilities fall that way (as a sibling comment points out.) But it's not "justified", because models don't have any ability to remember or cite sources, or even internally classify tokens as true or false (it's all just weights.) (And it's not a "belief", either, because there's no intentionality, but that's a more slippery concept.) Therefore, philosophically speaking, it is fundamentally impossible to acquire knowledge from a LLM unless you verify ("justify") a statement via some secondary, more trustworthy channel.

caturopath 3y ago
I tried to mention containing knowledge to avoid a discussion like this: not 'knowing' or 'having knowledge' or something that could lead to a discussion like this, which I don't think aligns with the point I was trying to make, which is that top LLMs are not merely good at things like grammar, but that a key part of what they do has to do about information about the world. FWIW, although 'justified true belief' is classic in the sense of being an old definition of knowledge, I don't think any significant number of modern philosophers would use it.
- lukev 3y ago
  Well, they're concerned with Gettier problems etc too. But yeah the problem what you mean by "knowing." For example, I am perfectly happy to say that a LLM "knows" words just like I "know" words, or that it "knows about" very popular topics in a loose sense. But that's a very different sense of "knows" from the kind of specific propositional knowledge we're usually interested in when using a LLM as a search engine, fact checker or source of information!
Filligree 3y ago
By that definition of knowledge, I don’t contain knowledge either. I’m clearly keeping track of how strongly I believe a variety of things. Not how I started believing them to begin with, however, so I lack the justification. And in a lot of cases, the lost justification would have been: “Because someone else said it, so now I’m aping them.”
- lukev 3y ago
  I mean, this is why philosophical skepticism is a thing. There's hundreds of books written on epistemology. But I think almost none of them would ascribe "knowledge" to a statistical engine that output propositions of random truth values weighted by how many examples of that proposition it's seen in it's training data. You can view the logprobs in the API and literally watch it choose to say different things based on it's internal dice rolls.
  - williamcotton 3y ago
    What we have that an LLM doesn’t have is a mobile body and other sensory inputs so we can verify knowledge through practice. Knowledge without a feedback mechanism for verification is prone to veer off course. Right now the feedback mechanism is based on human reinforcement. What if we put an LLM in an android’s body and let the machine interact with the world through the same senses that we do?
    - lukev 3y ago
      A body may be a necessary condition for epistemological knowledge, but it's certainly not sufficient... most animals are embodied but do not possess propositional knowledge (IMO anyway.) Ultimately though this is all about the definition of words. Humans find the distinction between "true" and "false" to be important and meaningful; animals don't care. AIs don't care (being incapable of caring), but humans care very much about the propositional veracity of statements that AI makes. The important thing is that we're on the same page about what's actually happening with these systems and how they work and don't slip into magical thinking or assume that just because a system is facile with language it automatically has a sense of truth or right or wrong. This is only going to get more complicated as models get more sophisticated.
      - ganzuul 3y ago
        > most animals are embodied but do not possess propositional knowledge https://www.youtube.com/watch?v=yhfl7kasjZc https://www.youtube.com/watch?v=yhfl7kasjZc There are a quite a few videos with the same reaction. There does seem to be something going on.

by lukev 3y ago

caturopath 3y ago

I tried to mention containing knowledge to avoid a discussion like this: not 'knowing' or 'having knowledge' or something that could lead to a discussion like this, which I don't think aligns with the point I was trying to make, which is that top LLMs are not merely good at things like grammar, but that a key part of what they do has to do about information about the world. FWIW, although 'justified true belief' is classic in the sense of being an old definition of knowledge, I don't think any significant number of modern philosophers would use it.

lukev 3y ago
Well, they're concerned with Gettier problems etc too. But yeah the problem what you mean by "knowing." For example, I am perfectly happy to say that a LLM "knows" words just like I "know" words, or that it "knows about" very popular topics in a loose sense. But that's a very different sense of "knows" from the kind of specific propositional knowledge we're usually interested in when using a LLM as a search engine, fact checker or source of information!

Filligree 3y ago

By that definition of knowledge, I don’t contain knowledge either. I’m clearly keeping track of how strongly I believe a variety of things. Not how I started believing them to begin with, however, so I lack the justification. And in a lot of cases, the lost justification would have been: “Because someone else said it, so now I’m aping them.”

lukev 3y ago
I mean, this is why philosophical skepticism is a thing. There's hundreds of books written on epistemology. But I think almost none of them would ascribe "knowledge" to a statistical engine that output propositions of random truth values weighted by how many examples of that proposition it's seen in it's training data. You can view the logprobs in the API and literally watch it choose to say different things based on it's internal dice rolls.
- williamcotton 3y ago
  What we have that an LLM doesn’t have is a mobile body and other sensory inputs so we can verify knowledge through practice. Knowledge without a feedback mechanism for verification is prone to veer off course. Right now the feedback mechanism is based on human reinforcement. What if we put an LLM in an android’s body and let the machine interact with the world through the same senses that we do?
  - lukev 3y ago
    A body may be a necessary condition for epistemological knowledge, but it's certainly not sufficient... most animals are embodied but do not possess propositional knowledge (IMO anyway.) Ultimately though this is all about the definition of words. Humans find the distinction between "true" and "false" to be important and meaningful; animals don't care. AIs don't care (being incapable of caring), but humans care very much about the propositional veracity of statements that AI makes. The important thing is that we're on the same page about what's actually happening with these systems and how they work and don't slip into magical thinking or assume that just because a system is facile with language it automatically has a sense of truth or right or wrong. This is only going to get more complicated as models get more sophisticated.
    - ganzuul 3y ago
      > most animals are embodied but do not possess propositional knowledge https://www.youtube.com/watch?v=yhfl7kasjZc https://www.youtube.com/watch?v=yhfl7kasjZc There are a quite a few videos with the same reaction. There does seem to be something going on.