11 ms·

> LLMs seem to operate on the principle of "will this pass for a correct answer" rather than "this is how this thing works, so here's a reasonable opinion answe

> LLMs seem to operate on the principle of "will this pass for a correct answer" rather than "this is how this thing works, so here's a reasonable opinion answering your question". Isn't the hint in the name? It's a language model, not a knowledge model. LLMs are exceptional at generating stuff that passes as coherent language (like, the parts of speech are all where one would expect them to be in the sentences). The trouble is people think it goes deeper when the knowledge modeling part only happens incidentally because language and knowledge modeling are so closely related.

mvdtnz 3y ago
> Isn't the hint in the name? It's a language model, not a knowledge model. Well yes, but how is this insight helpful? People aren't just looking at the name. Have your missed the last 10 months of ridiculous breathless coverage of LLMs? > The trouble is people think it goes deeper when the knowledge modeling part People are literally calling this shitty text prediction algorithm "AI". The discourse is completely out of hand.
- simonw 3y ago
  It is "AI", using the definition of that term that was established when the term was coined in 1956. https://en.m.wikipedia.org/wiki/Dartmouth_workshop https://en.m.wikipedia.org/wiki/Dartmouth_workshop > An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. LLMs fit that definition well. They're not AGI, but I think it's fair to call them AI.
  - mvdtnz 3y ago
    There is no widely accepted definition of "AI" and even if I accepted this one (I absolutely do not) no LLM meets this bar in the general sense.
    - trealira 3y ago
      Game tree heuristic searches, like the minimax algorithm with alpha-beta pruning, has also been called AI, which is why programs that play chess are called AI. What I think is the problem is that when people hear "artificial intelligence," they don't think of chess programs anymore; they think of something like Skynet from the Terminator. Or, maybe now they think of ChatGPT.
    - simonw 3y ago
      That 1956 workshop is widely credited as the place where the term AI was coined. If they don't get to define "AI", who does?
  - Izkata 3y ago
    What some people now call AGI, the average person thinks of when they hear AI.
AnimalMuppet 3y ago
> Isn't the hint in the name? It's a language model, not a knowledge model. True. But most of the training corpus was not linguistically-sound gibberish; it was actual text. There was knowledge encoded in the words. LLMs are large language models - large enough to have encoded some of the knowledge with the words. Some of. And they only encoded it. They didn't learn it, and they don't know it. It's just encoded in the words. It comes out sometimes in response to a prompt. Not always, not often enough to be relied on, but often enough to give users hope. What we really need is a language model coupled to a knowledge model.
- caturopath 3y ago
  > And they only encoded it. They didn't learn it, and they don't know it. People say stuff like this a lot, and I never know what to do with it. I'm not sure what specifically I'm supposed to get out of these distinctions.
  - j4yav 3y ago
    I'm not sure either. An alien could make a similar argument about our brains made of meat and electricity.
  - AnimalMuppet 3y ago
    Well, without bothering to get all epistemological about it, it's clear that LLMs doesn't know, say, the law in a way that a lawyer knows it, or physics in the way that a physicist knows it. And that's enough to be a problem, given how people are trying to use LLMs. They're trying to use it like a human who "knows" (whatever that means). And whatever an LLM actually does, it doesn't do that.
    - caturopath 3y ago
      My uncle doesn't know the law the way a lawyer knows it either, but that doesn't stop him from trying to use it like someone who does ;) If the claim is only "LLMs don't work like humans" and "LLMs make a lot of factual and logical errors", then that seems to be correct as far as it goes, but it doesn't go very far.
intended 3y ago
You would think. However, stating that LLMs predict text, not facts - and that LLMs don’t “think”, is enough to start a debate. Yeah the issue is anthropormphization and the “cool” factor of demos, vs actual experience.
- deleted 3y ago
  [deleted]
bsenftner 3y ago
> Isn't the hint in the name? Actually the hint is not in the name, that only tells a portion of the truth. If LLM were trained on textbooks teaching foreign languages from every other language, the name would suffice. But it does not, LLMs are trained on wide variety of texts, many of them social media, blog, and general web posts. The larger web is an ocean of bullshit, and that is where the bullshit within LLMs comes from. 'Textbooks are all you need' ought to be how LLMs are trained so people can use them for their "knowledge work" without bullshit ramifications.
- helsinkiandrew 3y ago
  > 'Textbooks are all you need' ought to be how LLMs are trained so people can use them for their "knowledge work" without bullshit ramifications. But in many spheres of knowledge there are still a lot of contradictory opinions in textbooks. Why did Rome fall?, What's the best X? In science there have been many schisms: quantum theory, tectonic plates, GMO/Organic etc. Writing code is probably one of the few areas where just about all content is 'good' - there may be a lot of bad or poor performing code in books and online but the vast majority of it 'works'.
  - bsenftner 3y ago
    Very good points. That gives me the idea of specifically training LLMs on each side of the active scientific controversies and then host/publish an event where they debate. Might be very interesting to geeks, or they will hate it. Probably both.
- OfSanguineFire 3y ago
  Textbooks are a rather bad source of info. Consider that history textbooks in many countries look at everything through the lens of a particular national mythology, since one of the goals of a country’s public education is to forge a cohesive society, and that is done by downplaying past controversies. For many other fields, introductory textbooks even at the university level may abound in oversimplifications, and it is only in later years of study that one begins, by reading specialized papers and monographs, to grasp how complicated the facts really are.
caturopath 3y ago
No, you're overindexing on the name. It doesn't mean that these LLMs are about linguistics and grammar specifically. Clearly, they aren't as likely to generate "Colorless green ideas sleep furiously" as "It's important to feed your dog". Obviously, they're better at grammar than many other things, but it's clear that they contain a vast amount of knowledge. They contain the information that the sky is blue and firetrucks are red, which isn't a matter of grammar or coherence, but is simply information about the world.
- capableweb 3y ago
  > It doesn't mean that these LLMs are about linguistics and grammar specifically. Correct. But it does mean they're about "language" rather than knowledge. It's meant to read, parse and generate language, no matter if it includes prose, facts, knowledge or whatever. > Clearly, they aren't as likely to generate "Colorless green ideas sleep furiously" as "It's important to feed your dog". Clearly, as the training set probably contains much more instances of the latter, than the first example. And if it's more probable to generate, that's the text it'll generate. It doesn't understand why or how, just that it's more likely, so that's the way it's going.
  - deleted 3y ago
    [deleted]
  - caturopath 3y ago
    > it does mean they're about "language" rather than knowledge Not really. It was invented because they were trained on natural language corpuses. It wasn't a deep, philosophical concept. It continues to be used for models that are doing completely non-languagy stuff, such as robotics, although I expect the growing use of 'foundation model' or some other less-confusing term will likely replace it.
  - FeepingCreature 3y ago
    Language is about knowledge. If it wasn't, what would we be doing here?
  - 3y ago
j4yav 3y ago
I can invent a grammar, explain it to an LLM, and it can apply it correctly in ambiguous situations. If its just generating output based on statistical probability based on its training corpus, how would that work? It feels like there's something emergent there that isn't just a very good probabilistic calculator.
Vegenoid 3y ago
> The trouble is people think it goes deeper when the knowledge modeling part only happens incidentally because language and knowledge modeling are so closely related. I think there's a strong argument to be made (that has been made) that much of our knowledge capability is built on language - that language did not simply allow us to communicate our knowledge, but actually allowed individuals to think in a new way. I certainly agree that current LLMs have no regard for truth and logic at their core, and there are clearly areas where their ability to read and write does not translate to producing true statements. LLMs can't build things out of language like humans can. I am less sure that LLMs don't have a long way to go before we reach the limits of what modeling language can do.

mvdtnz 3y ago

> Isn't the hint in the name? It's a language model, not a knowledge model. Well yes, but how is this insight helpful? People aren't just looking at the name. Have your missed the last 10 months of ridiculous breathless coverage of LLMs? > The trouble is people think it goes deeper when the knowledge modeling part People are literally calling this shitty text prediction algorithm "AI". The discourse is completely out of hand.

simonw 3y ago
It is "AI", using the definition of that term that was established when the term was coined in 1956. https://en.m.wikipedia.org/wiki/Dartmouth_workshop https://en.m.wikipedia.org/wiki/Dartmouth_workshop > An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. LLMs fit that definition well. They're not AGI, but I think it's fair to call them AI.
- mvdtnz 3y ago
  There is no widely accepted definition of "AI" and even if I accepted this one (I absolutely do not) no LLM meets this bar in the general sense.
  - trealira 3y ago
    Game tree heuristic searches, like the minimax algorithm with alpha-beta pruning, has also been called AI, which is why programs that play chess are called AI. What I think is the problem is that when people hear "artificial intelligence," they don't think of chess programs anymore; they think of something like Skynet from the Terminator. Or, maybe now they think of ChatGPT.
  - simonw 3y ago
    That 1956 workshop is widely credited as the place where the term AI was coined. If they don't get to define "AI", who does?
- Izkata 3y ago
  What some people now call AGI, the average person thinks of when they hear AI.

AnimalMuppet 3y ago

> Isn't the hint in the name? It's a language model, not a knowledge model. True. But most of the training corpus was not linguistically-sound gibberish; it was actual text. There was knowledge encoded in the words. LLMs are large language models - large enough to have encoded some of the knowledge with the words. Some of. And they only encoded it. They didn't learn it, and they don't know it. It's just encoded in the words. It comes out sometimes in response to a prompt. Not always, not often enough to be relied on, but often enough to give users hope. What we really need is a language model coupled to a knowledge model.

caturopath 3y ago
> And they only encoded it. They didn't learn it, and they don't know it. People say stuff like this a lot, and I never know what to do with it. I'm not sure what specifically I'm supposed to get out of these distinctions.
- j4yav 3y ago
  I'm not sure either. An alien could make a similar argument about our brains made of meat and electricity.
- AnimalMuppet 3y ago
  Well, without bothering to get all epistemological about it, it's clear that LLMs doesn't know, say, the law in a way that a lawyer knows it, or physics in the way that a physicist knows it. And that's enough to be a problem, given how people are trying to use LLMs. They're trying to use it like a human who "knows" (whatever that means). And whatever an LLM actually does, it doesn't do that.
  - caturopath 3y ago
    My uncle doesn't know the law the way a lawyer knows it either, but that doesn't stop him from trying to use it like someone who does ;) If the claim is only "LLMs don't work like humans" and "LLMs make a lot of factual and logical errors", then that seems to be correct as far as it goes, but it doesn't go very far.

intended 3y ago

You would think. However, stating that LLMs predict text, not facts - and that LLMs don’t “think”, is enough to start a debate. Yeah the issue is anthropormphization and the “cool” factor of demos, vs actual experience.

deleted 3y ago
[deleted]

bsenftner 3y ago

> Isn't the hint in the name? Actually the hint is not in the name, that only tells a portion of the truth. If LLM were trained on textbooks teaching foreign languages from every other language, the name would suffice. But it does not, LLMs are trained on wide variety of texts, many of them social media, blog, and general web posts. The larger web is an ocean of bullshit, and that is where the bullshit within LLMs comes from. 'Textbooks are all you need' ought to be how LLMs are trained so people can use them for their "knowledge work" without bullshit ramifications.

helsinkiandrew 3y ago
> 'Textbooks are all you need' ought to be how LLMs are trained so people can use them for their "knowledge work" without bullshit ramifications. But in many spheres of knowledge there are still a lot of contradictory opinions in textbooks. Why did Rome fall?, What's the best X? In science there have been many schisms: quantum theory, tectonic plates, GMO/Organic etc. Writing code is probably one of the few areas where just about all content is 'good' - there may be a lot of bad or poor performing code in books and online but the vast majority of it 'works'.
- bsenftner 3y ago
  Very good points. That gives me the idea of specifically training LLMs on each side of the active scientific controversies and then host/publish an event where they debate. Might be very interesting to geeks, or they will hate it. Probably both.
OfSanguineFire 3y ago
Textbooks are a rather bad source of info. Consider that history textbooks in many countries look at everything through the lens of a particular national mythology, since one of the goals of a country’s public education is to forge a cohesive society, and that is done by downplaying past controversies. For many other fields, introductory textbooks even at the university level may abound in oversimplifications, and it is only in later years of study that one begins, by reading specialized papers and monographs, to grasp how complicated the facts really are.

caturopath 3y ago

No, you're overindexing on the name. It doesn't mean that these LLMs are about linguistics and grammar specifically. Clearly, they aren't as likely to generate "Colorless green ideas sleep furiously" as "It's important to feed your dog". Obviously, they're better at grammar than many other things, but it's clear that they contain a vast amount of knowledge. They contain the information that the sky is blue and firetrucks are red, which isn't a matter of grammar or coherence, but is simply information about the world.

capableweb 3y ago
> It doesn't mean that these LLMs are about linguistics and grammar specifically. Correct. But it does mean they're about "language" rather than knowledge. It's meant to read, parse and generate language, no matter if it includes prose, facts, knowledge or whatever. > Clearly, they aren't as likely to generate "Colorless green ideas sleep furiously" as "It's important to feed your dog". Clearly, as the training set probably contains much more instances of the latter, than the first example. And if it's more probable to generate, that's the text it'll generate. It doesn't understand why or how, just that it's more likely, so that's the way it's going.
- deleted 3y ago
  [deleted]
- caturopath 3y ago
  > it does mean they're about "language" rather than knowledge Not really. It was invented because they were trained on natural language corpuses. It wasn't a deep, philosophical concept. It continues to be used for models that are doing completely non-languagy stuff, such as robotics, although I expect the growing use of 'foundation model' or some other less-confusing term will likely replace it.
- FeepingCreature 3y ago
  Language is about knowledge. If it wasn't, what would we be doing here?
- 3y ago

j4yav 3y ago

I can invent a grammar, explain it to an LLM, and it can apply it correctly in ambiguous situations. If its just generating output based on statistical probability based on its training corpus, how would that work? It feels like there's something emergent there that isn't just a very good probabilistic calculator.

Vegenoid 3y ago

> The trouble is people think it goes deeper when the knowledge modeling part only happens incidentally because language and knowledge modeling are so closely related. I think there's a strong argument to be made (that has been made) that much of our knowledge capability is built on language - that language did not simply allow us to communicate our knowledge, but actually allowed individuals to think in a new way. I certainly agree that current LLMs have no regard for truth and logic at their core, and there are clearly areas where their ability to read and write does not translate to producing true statements. LLMs can't build things out of language like humans can. I am less sure that LLMs don't have a long way to go before we reach the limits of what modeling language can do.