4 ms·
> It doesn't mean that these LLMs are about linguistics and grammar specifically. Correct. But it does mean they're about "language" rather than knowledge. It'
by capableweb 3y ago
> It doesn't mean that these LLMs are about linguistics and grammar specifically.
Correct. But it does mean they're about "language" rather than knowledge. It's meant to read, parse and generate language, no matter if it includes prose, facts, knowledge or whatever.
> Clearly, they aren't as likely to generate "Colorless green ideas sleep furiously" as "It's important to feed your dog".
Clearly, as the training set probably contains much more instances of the latter, than the first example. And if it's more probable to generate, that's the text it'll generate. It doesn't understand why or how, just that it's more likely, so that's the way it's going.
- deleted 3y ago[deleted]
- caturopath 3y ago> it does mean they're about "language" rather than knowledge Not really. It was invented because they were trained on natural language corpuses. It wasn't a deep, philosophical concept. It continues to be used for models that are doing completely non-languagy stuff, such as robotics, although I expect the growing use of 'foundation model' or some other less-confusing term will likely replace it.
- FeepingCreature 3y agoLanguage is about knowledge. If it wasn't, what would we be doing here?
- tbrownaw 3y ago> But it does mean they're about "language" rather than knowledge. To some extent, those are kind of the same thing.
- tessierashpool 3y agothey really are not. you can use language as a proxy for knowledge in some cases, but doing so guarantees generating bullshit, which is the entire point of the OP.
- tivert 3y ago> To some extent, those are kind of the same thing. They are not the same thing. They are not even "kind of" the same thing, for any reasonable definition of "kind of." That should be obvious, because a lie is language, which is pretty close to the opposite of knowledge. I mean no offense, but your hedging words are doing so much work that I think your statement counts as bullshit. Edit: thinking about it more, I think your comment shows the particular kind of bullshit that often characterizes tech people's thinking and is so useful for hyping shit (e.g. take something hard, lossily reformulate it into something easier but vaguely similar, solve the easy thing, then declare you solved the hard thing).
- tbrownaw 3y agoA language is a particular way of serializing concepts and their relationships. Knowledge is... concepts and their relationships, typically with the implication of being grounded in objective reality (although one can be knowledgeable about things like, say, Tolkien, or Star Wars, or fantasy tropes more generally). . Language is (a particular way of representing) knowledge, in about the same way that a particular .cpp file is (a particular way of representing) a linked list or whatever.
- deleted 3y ago[deleted]
- tivert 3y ago> Language is (a particular way of representing) knowledge, in about the same way that a particular .cpp file is (a particular way of representing) a linked list or whatever. So, in your view, are .cpp files and linked lists "kind of" the same thing? Language can be used to express some kinds of knowledge, but they're not even "kind of" synonymous. Language can express a lot of things that aren't knowledge, and there's knowledge that can't be expressed with language (qualia, at least).
- TeMPOraL 3y agoOf course they are the same thing. Language as humans use it isn't a god-given perfect formal grammar system. It's people grunting and gulping about what they experience, for many thousand of years, each generation picking up the patterns from the generation before, and adding to them. Language was, is, and always has been about knowledge, because you literally build the language skill up from correlations. All the more and less formal models we invent for describing "language"? This is just fitting curves to a snapshot of reality. So sure, "a lie is language, which is pretty close to the opposite of knowledge" - a lie can be expressed in the formal model. But the truth (as subjectively understood) is much more likely to be spoken or written down in practice. Communication isn't random. Language isn't random. Learning it is picking up the patterns.
- QuantumGood 3y agoit does mean they're about "language" rather than knowledge Language in part encodes knowledge.
- smoldesu 3y agoIt also encodes opinions, fantasy, bad-faith arguments and lies without discrimination. Given that LLMs aren't trained to separate their sources, I think it's fair to say today's AI focuses more on language than the integrity of it's answer.
- QuantumGood 3y agoAgreed, discriminating what language contains is the issue.