4 ms·

No, you're overindexing on the name. It doesn't mean that these LLMs are about linguistics and grammar specifically. Clearly, they aren't as likely to generate

No, you're overindexing on the name. It doesn't mean that these LLMs are about linguistics and grammar specifically. Clearly, they aren't as likely to generate "Colorless green ideas sleep furiously" as "It's important to feed your dog". Obviously, they're better at grammar than many other things, but it's clear that they contain a vast amount of knowledge. They contain the information that the sky is blue and firetrucks are red, which isn't a matter of grammar or coherence, but is simply information about the world.

capableweb 3y ago
> It doesn't mean that these LLMs are about linguistics and grammar specifically. Correct. But it does mean they're about "language" rather than knowledge. It's meant to read, parse and generate language, no matter if it includes prose, facts, knowledge or whatever. > Clearly, they aren't as likely to generate "Colorless green ideas sleep furiously" as "It's important to feed your dog". Clearly, as the training set probably contains much more instances of the latter, than the first example. And if it's more probable to generate, that's the text it'll generate. It doesn't understand why or how, just that it's more likely, so that's the way it's going.
- deleted 3y ago
  [deleted]
- caturopath 3y ago
  > it does mean they're about "language" rather than knowledge Not really. It was invented because they were trained on natural language corpuses. It wasn't a deep, philosophical concept. It continues to be used for models that are doing completely non-languagy stuff, such as robotics, although I expect the growing use of 'foundation model' or some other less-confusing term will likely replace it.
- FeepingCreature 3y ago
  Language is about knowledge. If it wasn't, what would we be doing here?
- tbrownaw 3y ago
  > But it does mean they're about "language" rather than knowledge. To some extent, those are kind of the same thing.
  - tessierashpool 3y ago
    they really are not. you can use language as a proxy for knowledge in some cases, but doing so guarantees generating bullshit, which is the entire point of the OP.
  - tivert 3y ago
    > To some extent, those are kind of the same thing. They are not the same thing. They are not even "kind of" the same thing, for any reasonable definition of "kind of." That should be obvious, because a lie is language, which is pretty close to the opposite of knowledge. I mean no offense, but your hedging words are doing so much work that I think your statement counts as bullshit. Edit: thinking about it more, I think your comment shows the particular kind of bullshit that often characterizes tech people's thinking and is so useful for hyping shit (e.g. take something hard, lossily reformulate it into something easier but vaguely similar, solve the easy thing, then declare you solved the hard thing).
    - tbrownaw 3y ago
      A language is a particular way of serializing concepts and their relationships. Knowledge is... concepts and their relationships, typically with the implication of being grounded in objective reality (although one can be knowledgeable about things like, say, Tolkien, or Star Wars, or fantasy tropes more generally). . Language is (a particular way of representing) knowledge, in about the same way that a particular .cpp file is (a particular way of representing) a linked list or whatever.
      - deleted 3y ago
        [deleted]
      - tivert 3y ago
        > Language is (a particular way of representing) knowledge, in about the same way that a particular .cpp file is (a particular way of representing) a linked list or whatever. So, in your view, are .cpp files and linked lists "kind of" the same thing? Language can be used to express some kinds of knowledge, but they're not even "kind of" synonymous. Language can express a lot of things that aren't knowledge, and there's knowledge that can't be expressed with language (qualia, at least).
- QuantumGood 3y ago
  it does mean they're about "language" rather than knowledge Language in part encodes knowledge.
  - smoldesu 3y ago
    It also encodes opinions, fantasy, bad-faith arguments and lies without discrimination. Given that LLMs aren't trained to separate their sources, I think it's fair to say today's AI focuses more on language than the integrity of it's answer.
    - QuantumGood 3y ago
      Agreed, discriminating what language contains is the issue.
lukev 3y ago
I would argue they do not contain knowledge using the classic definition of knowledge as "justified true belief." They can emit statements that happen to be true because the probabilities fall that way (as a sibling comment points out.) But it's not "justified", because models don't have any ability to remember or cite sources, or even internally classify tokens as true or false (it's all just weights.) (And it's not a "belief", either, because there's no intentionality, but that's a more slippery concept.) Therefore, philosophically speaking, it is fundamentally impossible to acquire knowledge from a LLM unless you verify ("justify") a statement via some secondary, more trustworthy channel.
- caturopath 3y ago
  I tried to mention containing knowledge to avoid a discussion like this: not 'knowing' or 'having knowledge' or something that could lead to a discussion like this, which I don't think aligns with the point I was trying to make, which is that top LLMs are not merely good at things like grammar, but that a key part of what they do has to do about information about the world. FWIW, although 'justified true belief' is classic in the sense of being an old definition of knowledge, I don't think any significant number of modern philosophers would use it.
  - lukev 3y ago
    Well, they're concerned with Gettier problems etc too. But yeah the problem what you mean by "knowing." For example, I am perfectly happy to say that a LLM "knows" words just like I "know" words, or that it "knows about" very popular topics in a loose sense. But that's a very different sense of "knows" from the kind of specific propositional knowledge we're usually interested in when using a LLM as a search engine, fact checker or source of information!
- Filligree 3y ago
  By that definition of knowledge, I don’t contain knowledge either. I’m clearly keeping track of how strongly I believe a variety of things. Not how I started believing them to begin with, however, so I lack the justification. And in a lot of cases, the lost justification would have been: “Because someone else said it, so now I’m aping them.”
gorjusborg 3y ago
> but it's clear that they contain a vast amount of knowledge. Is it though? There's nuance in what we call knowledge, but to me, knowledge is not uni-dimensional. What I mean by that is, I know when I _know_ something, and I have learned that that is not a trivial skill. How do I know I _know_ something? Usually because I have searched for evidence from multiple angles / dimensions / senses. When multiple independent observations of quality agree, and my conclusion matches the result, it is safe to say that I know that fact. An LLM may have multiple 'dimensions' in a linear algebra sense, but does it have independent information?
- caturopath 3y ago
  For what it's worth, I mentioned they _contain_ the knowledge to try to avoid discussions about whether they really "know" anything -- if there's a better term like 'information about the world' or something, I think that's all I was trying to tell the original person I replied to. The point wasn't that they do what humans do, but that they are not just grammar machines. > How do I know I _know_ something? Usually because I have searched for evidence from multiple angles / dimensions / senses. When multiple independent observations of quality agree, and my conclusion matches the result, it is safe to say that I know that fact. I wanted to thank you for laying out this perspective. In my experience, in discussions about LLMs, a lot of time people get caught up on words like 'know' which work well for talking about humans, but don't cash them out in definitions I know what to do with. This is a really content-ful of sharing what you mean by your claim.
- TeMPOraL 3y ago
  > How do I know I _know_ something? Usually because I have searched for evidence from multiple angles / dimensions / senses. When multiple independent observations of quality agree, and my conclusion matches the result, it is safe to say that I know that fact. This is technically true, but no one actually does much of it consciously. But this is how you picked up every skill and understanding, including language, since you've been born. And this - the unconscious fusion of correlated patterns - is pretty much what LLMs undergo in training. EDIT: Also, a lot of confusion is created by using terms like "a fact" or "a conclusion" or "a result", as if they were fixed points in space, fully defined mathematical symbols. But there exist no such thing. There is no binary "know/don't know", there is no binary "result match/does't match the conclusion". Those are all continuous quantities, that get rounded off for convenience.
emporas 3y ago
>Clearly, they aren't as likely to generate "Colorless green ideas sleep furiously" as "It's important to feed your dog". There is a setting right there called temperature, no?

capableweb 3y ago

> It doesn't mean that these LLMs are about linguistics and grammar specifically. Correct. But it does mean they're about "language" rather than knowledge. It's meant to read, parse and generate language, no matter if it includes prose, facts, knowledge or whatever. > Clearly, they aren't as likely to generate "Colorless green ideas sleep furiously" as "It's important to feed your dog". Clearly, as the training set probably contains much more instances of the latter, than the first example. And if it's more probable to generate, that's the text it'll generate. It doesn't understand why or how, just that it's more likely, so that's the way it's going.

deleted 3y ago
[deleted]
caturopath 3y ago
> it does mean they're about "language" rather than knowledge Not really. It was invented because they were trained on natural language corpuses. It wasn't a deep, philosophical concept. It continues to be used for models that are doing completely non-languagy stuff, such as robotics, although I expect the growing use of 'foundation model' or some other less-confusing term will likely replace it.
FeepingCreature 3y ago
Language is about knowledge. If it wasn't, what would we be doing here?
tbrownaw 3y ago
> But it does mean they're about "language" rather than knowledge. To some extent, those are kind of the same thing.
- tessierashpool 3y ago
  they really are not. you can use language as a proxy for knowledge in some cases, but doing so guarantees generating bullshit, which is the entire point of the OP.
- tivert 3y ago
  > To some extent, those are kind of the same thing. They are not the same thing. They are not even "kind of" the same thing, for any reasonable definition of "kind of." That should be obvious, because a lie is language, which is pretty close to the opposite of knowledge. I mean no offense, but your hedging words are doing so much work that I think your statement counts as bullshit. Edit: thinking about it more, I think your comment shows the particular kind of bullshit that often characterizes tech people's thinking and is so useful for hyping shit (e.g. take something hard, lossily reformulate it into something easier but vaguely similar, solve the easy thing, then declare you solved the hard thing).
  - tbrownaw 3y ago
    A language is a particular way of serializing concepts and their relationships. Knowledge is... concepts and their relationships, typically with the implication of being grounded in objective reality (although one can be knowledgeable about things like, say, Tolkien, or Star Wars, or fantasy tropes more generally). . Language is (a particular way of representing) knowledge, in about the same way that a particular .cpp file is (a particular way of representing) a linked list or whatever.
    - deleted 3y ago
      [deleted]
    - tivert 3y ago
      > Language is (a particular way of representing) knowledge, in about the same way that a particular .cpp file is (a particular way of representing) a linked list or whatever. So, in your view, are .cpp files and linked lists "kind of" the same thing? Language can be used to express some kinds of knowledge, but they're not even "kind of" synonymous. Language can express a lot of things that aren't knowledge, and there's knowledge that can't be expressed with language (qualia, at least).
QuantumGood 3y ago
it does mean they're about "language" rather than knowledge Language in part encodes knowledge.
- smoldesu 3y ago
  It also encodes opinions, fantasy, bad-faith arguments and lies without discrimination. Given that LLMs aren't trained to separate their sources, I think it's fair to say today's AI focuses more on language than the integrity of it's answer.
  - QuantumGood 3y ago
    Agreed, discriminating what language contains is the issue.

lukev 3y ago

I would argue they do not contain knowledge using the classic definition of knowledge as "justified true belief." They can emit statements that happen to be true because the probabilities fall that way (as a sibling comment points out.) But it's not "justified", because models don't have any ability to remember or cite sources, or even internally classify tokens as true or false (it's all just weights.) (And it's not a "belief", either, because there's no intentionality, but that's a more slippery concept.) Therefore, philosophically speaking, it is fundamentally impossible to acquire knowledge from a LLM unless you verify ("justify") a statement via some secondary, more trustworthy channel.

caturopath 3y ago
I tried to mention containing knowledge to avoid a discussion like this: not 'knowing' or 'having knowledge' or something that could lead to a discussion like this, which I don't think aligns with the point I was trying to make, which is that top LLMs are not merely good at things like grammar, but that a key part of what they do has to do about information about the world. FWIW, although 'justified true belief' is classic in the sense of being an old definition of knowledge, I don't think any significant number of modern philosophers would use it.
- lukev 3y ago
  Well, they're concerned with Gettier problems etc too. But yeah the problem what you mean by "knowing." For example, I am perfectly happy to say that a LLM "knows" words just like I "know" words, or that it "knows about" very popular topics in a loose sense. But that's a very different sense of "knows" from the kind of specific propositional knowledge we're usually interested in when using a LLM as a search engine, fact checker or source of information!
Filligree 3y ago
By that definition of knowledge, I don’t contain knowledge either. I’m clearly keeping track of how strongly I believe a variety of things. Not how I started believing them to begin with, however, so I lack the justification. And in a lot of cases, the lost justification would have been: “Because someone else said it, so now I’m aping them.”

gorjusborg 3y ago

> but it's clear that they contain a vast amount of knowledge. Is it though? There's nuance in what we call knowledge, but to me, knowledge is not uni-dimensional. What I mean by that is, I know when I _know_ something, and I have learned that that is not a trivial skill. How do I know I _know_ something? Usually because I have searched for evidence from multiple angles / dimensions / senses. When multiple independent observations of quality agree, and my conclusion matches the result, it is safe to say that I know that fact. An LLM may have multiple 'dimensions' in a linear algebra sense, but does it have independent information?

caturopath 3y ago
For what it's worth, I mentioned they _contain_ the knowledge to try to avoid discussions about whether they really "know" anything -- if there's a better term like 'information about the world' or something, I think that's all I was trying to tell the original person I replied to. The point wasn't that they do what humans do, but that they are not just grammar machines. > How do I know I _know_ something? Usually because I have searched for evidence from multiple angles / dimensions / senses. When multiple independent observations of quality agree, and my conclusion matches the result, it is safe to say that I know that fact. I wanted to thank you for laying out this perspective. In my experience, in discussions about LLMs, a lot of time people get caught up on words like 'know' which work well for talking about humans, but don't cash them out in definitions I know what to do with. This is a really content-ful of sharing what you mean by your claim.
TeMPOraL 3y ago
> How do I know I _know_ something? Usually because I have searched for evidence from multiple angles / dimensions / senses. When multiple independent observations of quality agree, and my conclusion matches the result, it is safe to say that I know that fact. This is technically true, but no one actually does much of it consciously. But this is how you picked up every skill and understanding, including language, since you've been born. And this - the unconscious fusion of correlated patterns - is pretty much what LLMs undergo in training. EDIT: Also, a lot of confusion is created by using terms like "a fact" or "a conclusion" or "a result", as if they were fixed points in space, fully defined mathematical symbols. But there exist no such thing. There is no binary "know/don't know", there is no binary "result match/does't match the conclusion". Those are all continuous quantities, that get rounded off for convenience.

emporas 3y ago

>Clearly, they aren't as likely to generate "Colorless green ideas sleep furiously" as "It's important to feed your dog". There is a setting right there called temperature, no?