4 ms·

If this proves LLMs are not intelligent, then when I ask ChatGPT or Gemini and they give me the correct answer, does that mean those are not LLMs?

ksaj 3y ago
It may very well mean it has seen that question answered before. It'll be hard to find a question that nobody has ever asked or discussed on the Internet, but if you can think of one, ask an LLM and see if it can figure it out correctly.
- zeroCalories 3y ago
  I asked Gemini this. Pretty unlikely it has seen this exact question before, but it answered it correctly. Today is February 29 2024. What will tomorrow be? Since today, February 29, 2024, is a leap day, tomorrow will be March 1, 2024. This is because leap years, which occur every four years (except for specific exceptions), add an extra day to February to better align the calendar with the Earth's revolution around the sun. As today is the added day, the following day becomes March 1st.
  - mannykannot 3y ago
    It is likely it has seen similar question-answer pairs, and it seems entirely plausible that substituting 2024 where a different year appeared scores highly on the most-probable-next-token measure, based only on those similar examples. The same caveat as on my previous reply applies here.
    - zeroCalories 3y ago
      That's fine. I'm simply stating that an inability of one particular(locally ran 7b) LLM to answer a question does not disprove the intelligence of LLMs. But to answer your point, if the statistical model is complex enough, is that not intelligence? We could keep playing this game, and at some point the LLM might mess up, but I imagine we can get pretty far with the best models available.
      - mannykannot 3y ago
        > if the statistical model is complex enough, is that not intelligence? Well, that is the question. I have speculated a bit more about that in these threads and elsewhere. One of the issues is whether a vast knowledge of the statistics drawn from language use can lead to an understanding that the language is about an external world. https://news.ycombinator.com/item?id=39501090 https://news.ycombinator.com/item?id=39501090
        zeroCalories 3y ago
        It's doubtful that any language can fully express the medium in which it is expressed. But that shouldn't discourage us about an LLM's intelligence, as the sense data you get about the world is, in a sense, also a language. An LLM has a different view of the world and reality than us, but it still has a view.
        mannykannot 3y ago
        > It's doubtful that any language can fully express the medium in which it is expressed. I feel I am missing the point here, maybe because I see language as being substrate-independent: while it needs some medium in which to be expressed, any medium which allows for binary distinctions is sufficient. The medium seems to me to be an issue independent of the fact that while language is a formal system with complex and not always consistent rules, it is also that case that, in most of human language use (and especially everyday use), it is about a world external to the speaker/writer and audience/reader. It also seem to me that, given the way LLMs are constructed and trained, they gain a good grasp of the formal properties of language (meaning the statistical properties of language as it is actually used, not just its grammar.) Does it also grasp the about-the-world aspect of language? I don't think we rationally have to assume the former leads to the latter, and we should be skeptical, firstly because we know how LLMs are made and secondly our prior experiences with language use has always been with people, for whom the relationship between language and the real world is so clear and ubiquitous that it is hard for us to consider these aspects separately, possibly leading us to anthropomorphize LLMs. If they do grasp the about-the-world aspect of language, then it seems to me just a short, and perhaps inevitable, step from there to self-awareness - seeing oneself as an individual in that world - and that would be a very big deal, IMHO.
    - causal 3y ago
      This really misrepresents what is happening in these models. Complex semantics are represented in the latent space. Relationships between embeddings have meanings we're able to sample to get these probabilistic strings - the fact that probabilities are involved does not mean there is no understanding behind the output.
      - mannykannot 3y ago
        > Complex semantics are represented in the latent space. In this particular case, what's going on doesn't seem all that complex. > The fact that probabilities are involved does not mean there is no understanding behind the output. And that, in turn, does not mean that there is - which is the question of interest here. The burden of proof is on anyone claiming to know the answer, either way.
mannykannot 3y ago
Ordinary, pre-LLM databases are not regarded by anyone as intelligent (AFAIK), but you can still get correct answers from them. Therefore, giving right answers is not a defining characteristic of intelligence. Note that this reply does not endorse any particular position on the question of whether LLMs have any sort of intelligence.
root_axis 3y ago
When a calculator gives me the correct answer does that prove it's intelligent?
- zeroCalories 3y ago
  No, but when a human gives you an incorrect answer it does not mean all humans are unintelligent.

ksaj 3y ago

It may very well mean it has seen that question answered before. It'll be hard to find a question that nobody has ever asked or discussed on the Internet, but if you can think of one, ask an LLM and see if it can figure it out correctly.

zeroCalories 3y ago
I asked Gemini this. Pretty unlikely it has seen this exact question before, but it answered it correctly. Today is February 29 2024. What will tomorrow be? Since today, February 29, 2024, is a leap day, tomorrow will be March 1, 2024. This is because leap years, which occur every four years (except for specific exceptions), add an extra day to February to better align the calendar with the Earth's revolution around the sun. As today is the added day, the following day becomes March 1st.
- mannykannot 3y ago
  It is likely it has seen similar question-answer pairs, and it seems entirely plausible that substituting 2024 where a different year appeared scores highly on the most-probable-next-token measure, based only on those similar examples. The same caveat as on my previous reply applies here.
  - zeroCalories 3y ago
    That's fine. I'm simply stating that an inability of one particular(locally ran 7b) LLM to answer a question does not disprove the intelligence of LLMs. But to answer your point, if the statistical model is complex enough, is that not intelligence? We could keep playing this game, and at some point the LLM might mess up, but I imagine we can get pretty far with the best models available.
    - mannykannot 3y ago
      > if the statistical model is complex enough, is that not intelligence? Well, that is the question. I have speculated a bit more about that in these threads and elsewhere. One of the issues is whether a vast knowledge of the statistics drawn from language use can lead to an understanding that the language is about an external world. https://news.ycombinator.com/item?id=39501090 https://news.ycombinator.com/item?id=39501090
      - zeroCalories 3y ago
        It's doubtful that any language can fully express the medium in which it is expressed. But that shouldn't discourage us about an LLM's intelligence, as the sense data you get about the world is, in a sense, also a language. An LLM has a different view of the world and reality than us, but it still has a view.
        mannykannot 3y ago
        > It's doubtful that any language can fully express the medium in which it is expressed. I feel I am missing the point here, maybe because I see language as being substrate-independent: while it needs some medium in which to be expressed, any medium which allows for binary distinctions is sufficient. The medium seems to me to be an issue independent of the fact that while language is a formal system with complex and not always consistent rules, it is also that case that, in most of human language use (and especially everyday use), it is about a world external to the speaker/writer and audience/reader. It also seem to me that, given the way LLMs are constructed and trained, they gain a good grasp of the formal properties of language (meaning the statistical properties of language as it is actually used, not just its grammar.) Does it also grasp the about-the-world aspect of language? I don't think we rationally have to assume the former leads to the latter, and we should be skeptical, firstly because we know how LLMs are made and secondly our prior experiences with language use has always been with people, for whom the relationship between language and the real world is so clear and ubiquitous that it is hard for us to consider these aspects separately, possibly leading us to anthropomorphize LLMs. If they do grasp the about-the-world aspect of language, then it seems to me just a short, and perhaps inevitable, step from there to self-awareness - seeing oneself as an individual in that world - and that would be a very big deal, IMHO.
  - causal 3y ago
    This really misrepresents what is happening in these models. Complex semantics are represented in the latent space. Relationships between embeddings have meanings we're able to sample to get these probabilistic strings - the fact that probabilities are involved does not mean there is no understanding behind the output.
    - mannykannot 3y ago
      > Complex semantics are represented in the latent space. In this particular case, what's going on doesn't seem all that complex. > The fact that probabilities are involved does not mean there is no understanding behind the output. And that, in turn, does not mean that there is - which is the question of interest here. The burden of proof is on anyone claiming to know the answer, either way.

mannykannot 3y ago

Ordinary, pre-LLM databases are not regarded by anyone as intelligent (AFAIK), but you can still get correct answers from them. Therefore, giving right answers is not a defining characteristic of intelligence. Note that this reply does not endorse any particular position on the question of whether LLMs have any sort of intelligence.

root_axis 3y ago

When a calculator gives me the correct answer does that prove it's intelligent?

zeroCalories 3y ago
No, but when a human gives you an incorrect answer it does not mean all humans are unintelligent.