4 ms·

That's fine. I'm simply stating that an inability of one particular(locally ran 7b) LLM to answer a question does not disprove the intelligence of LLMs. But to

That's fine. I'm simply stating that an inability of one particular(locally ran 7b) LLM to answer a question does not disprove the intelligence of LLMs. But to answer your point, if the statistical model is complex enough, is that not intelligence? We could keep playing this game, and at some point the LLM might mess up, but I imagine we can get pretty far with the best models available.

mannykannot 3y ago
> if the statistical model is complex enough, is that not intelligence? Well, that is the question. I have speculated a bit more about that in these threads and elsewhere. One of the issues is whether a vast knowledge of the statistics drawn from language use can lead to an understanding that the language is about an external world. https://news.ycombinator.com/item?id=39501090 https://news.ycombinator.com/item?id=39501090
- zeroCalories 3y ago
  It's doubtful that any language can fully express the medium in which it is expressed. But that shouldn't discourage us about an LLM's intelligence, as the sense data you get about the world is, in a sense, also a language. An LLM has a different view of the world and reality than us, but it still has a view.
  - mannykannot 3y ago
    > It's doubtful that any language can fully express the medium in which it is expressed. I feel I am missing the point here, maybe because I see language as being substrate-independent: while it needs some medium in which to be expressed, any medium which allows for binary distinctions is sufficient. The medium seems to me to be an issue independent of the fact that while language is a formal system with complex and not always consistent rules, it is also that case that, in most of human language use (and especially everyday use), it is about a world external to the speaker/writer and audience/reader. It also seem to me that, given the way LLMs are constructed and trained, they gain a good grasp of the formal properties of language (meaning the statistical properties of language as it is actually used, not just its grammar.) Does it also grasp the about-the-world aspect of language? I don't think we rationally have to assume the former leads to the latter, and we should be skeptical, firstly because we know how LLMs are made and secondly our prior experiences with language use has always been with people, for whom the relationship between language and the real world is so clear and ubiquitous that it is hard for us to consider these aspects separately, possibly leading us to anthropomorphize LLMs. If they do grasp the about-the-world aspect of language, then it seems to me just a short, and perhaps inevitable, step from there to self-awareness - seeing oneself as an individual in that world - and that would be a very big deal, IMHO.
    - zeroCalories 3y ago
      When I say doubtful, I mean in a practical sense. If you view the world as a collection of facts, describing even mundane things fully requires an immense number of facts. You require an abstraction, and it's doubtful that a useful abstraction won't wash away details about the world. I also think you're making an arbitrary distinction between human input data, and LLM input data. I could ask the exact same questions about a human's understanding of the real world. All I can say about both humans and LLMs is that they seem to both make good predictions.
      - mannykannot 3y ago
        > If you view the world as a collection of facts, describing even mundane things fully requires an immense number of facts... We already have some answers here: the resources needed by current LLMs sets an upper bound on what is needed to do what they do, and we can also make a reasonable estimate of the resources used by the human brain to do what it does. In the case of LLMs, there is no mystery about how those resources are put to use, as they were designed and implemented by humans (we may be surprised by how much they can do with those resources being used the way they are - well, I certainly was - but that is not the same thing as the way they use them being a mystery, which is essentially where we are with respect to human brains.) > I could ask the exact same questions about a human's understanding of the real world... It is already well-established that by the age of two, infants are developing a rudimentary theory of mind - an understanding that other people have minds - and by around four, they begin to grasp that thoughts in the mind may not be true [1]. These abilities seem to require the recognition of an external world as a prerequisite. Do LLMs give any indication of doing so? I'm not up-to-date on the research, but the last paper I saw on the topic only seemed to demonstrate that they can sometimes produce sentences as if they did - but, given the way their sentences are generated, it is difficult to say that this means anything more than that these sentences are the sort of sentences that a human is likely to say in the same situation. Update: this point just occurred to me: LLMs receive tokens, not words. Words have real-world semantics, but, in general, tokens do not. To me, this increases my doubt as to whether LLMs could understand that language is about an external world. [1] https://www.child-encyclopedia.com/social-cognition/according-experts/development-theory-mind-early-childhood https://www.child-encyclopedia.com/social-cognition/accordin...

mannykannot 3y ago

> if the statistical model is complex enough, is that not intelligence? Well, that is the question. I have speculated a bit more about that in these threads and elsewhere. One of the issues is whether a vast knowledge of the statistics drawn from language use can lead to an understanding that the language is about an external world. https://news.ycombinator.com/item?id=39501090 https://news.ycombinator.com/item?id=39501090

zeroCalories 3y ago
It's doubtful that any language can fully express the medium in which it is expressed. But that shouldn't discourage us about an LLM's intelligence, as the sense data you get about the world is, in a sense, also a language. An LLM has a different view of the world and reality than us, but it still has a view.
- mannykannot 3y ago
  > It's doubtful that any language can fully express the medium in which it is expressed. I feel I am missing the point here, maybe because I see language as being substrate-independent: while it needs some medium in which to be expressed, any medium which allows for binary distinctions is sufficient. The medium seems to me to be an issue independent of the fact that while language is a formal system with complex and not always consistent rules, it is also that case that, in most of human language use (and especially everyday use), it is about a world external to the speaker/writer and audience/reader. It also seem to me that, given the way LLMs are constructed and trained, they gain a good grasp of the formal properties of language (meaning the statistical properties of language as it is actually used, not just its grammar.) Does it also grasp the about-the-world aspect of language? I don't think we rationally have to assume the former leads to the latter, and we should be skeptical, firstly because we know how LLMs are made and secondly our prior experiences with language use has always been with people, for whom the relationship between language and the real world is so clear and ubiquitous that it is hard for us to consider these aspects separately, possibly leading us to anthropomorphize LLMs. If they do grasp the about-the-world aspect of language, then it seems to me just a short, and perhaps inevitable, step from there to self-awareness - seeing oneself as an individual in that world - and that would be a very big deal, IMHO.
  - zeroCalories 3y ago
    When I say doubtful, I mean in a practical sense. If you view the world as a collection of facts, describing even mundane things fully requires an immense number of facts. You require an abstraction, and it's doubtful that a useful abstraction won't wash away details about the world. I also think you're making an arbitrary distinction between human input data, and LLM input data. I could ask the exact same questions about a human's understanding of the real world. All I can say about both humans and LLMs is that they seem to both make good predictions.
    - mannykannot 3y ago
      > If you view the world as a collection of facts, describing even mundane things fully requires an immense number of facts... We already have some answers here: the resources needed by current LLMs sets an upper bound on what is needed to do what they do, and we can also make a reasonable estimate of the resources used by the human brain to do what it does. In the case of LLMs, there is no mystery about how those resources are put to use, as they were designed and implemented by humans (we may be surprised by how much they can do with those resources being used the way they are - well, I certainly was - but that is not the same thing as the way they use them being a mystery, which is essentially where we are with respect to human brains.) > I could ask the exact same questions about a human's understanding of the real world... It is already well-established that by the age of two, infants are developing a rudimentary theory of mind - an understanding that other people have minds - and by around four, they begin to grasp that thoughts in the mind may not be true [1]. These abilities seem to require the recognition of an external world as a prerequisite. Do LLMs give any indication of doing so? I'm not up-to-date on the research, but the last paper I saw on the topic only seemed to demonstrate that they can sometimes produce sentences as if they did - but, given the way their sentences are generated, it is difficult to say that this means anything more than that these sentences are the sort of sentences that a human is likely to say in the same situation. Update: this point just occurred to me: LLMs receive tokens, not words. Words have real-world semantics, but, in general, tokens do not. To me, this increases my doubt as to whether LLMs could understand that language is about an external world. [1] https://www.child-encyclopedia.com/social-cognition/according-experts/development-theory-mind-early-childhood https://www.child-encyclopedia.com/social-cognition/accordin...