7 ms·

I asked Gemini this. Pretty unlikely it has seen this exact question before, but it answered it correctly. Today is February 29 2024. What will tomorrow be? S

I asked Gemini this. Pretty unlikely it has seen this exact question before, but it answered it correctly. Today is February 29 2024. What will tomorrow be? Since today, February 29, 2024, is a leap day, tomorrow will be March 1, 2024. This is because leap years, which occur every four years (except for specific exceptions), add an extra day to February to better align the calendar with the Earth's revolution around the sun. As today is the added day, the following day becomes March 1st.

mannykannot 3y ago
It is likely it has seen similar question-answer pairs, and it seems entirely plausible that substituting 2024 where a different year appeared scores highly on the most-probable-next-token measure, based only on those similar examples. The same caveat as on my previous reply applies here.
- zeroCalories 3y ago
  That's fine. I'm simply stating that an inability of one particular(locally ran 7b) LLM to answer a question does not disprove the intelligence of LLMs. But to answer your point, if the statistical model is complex enough, is that not intelligence? We could keep playing this game, and at some point the LLM might mess up, but I imagine we can get pretty far with the best models available.
  - mannykannot 3y ago
    > if the statistical model is complex enough, is that not intelligence? Well, that is the question. I have speculated a bit more about that in these threads and elsewhere. One of the issues is whether a vast knowledge of the statistics drawn from language use can lead to an understanding that the language is about an external world. https://news.ycombinator.com/item?id=39501090 https://news.ycombinator.com/item?id=39501090
    - zeroCalories 3y ago
      It's doubtful that any language can fully express the medium in which it is expressed. But that shouldn't discourage us about an LLM's intelligence, as the sense data you get about the world is, in a sense, also a language. An LLM has a different view of the world and reality than us, but it still has a view.
      - mannykannot 3y ago
        > It's doubtful that any language can fully express the medium in which it is expressed. I feel I am missing the point here, maybe because I see language as being substrate-independent: while it needs some medium in which to be expressed, any medium which allows for binary distinctions is sufficient. The medium seems to me to be an issue independent of the fact that while language is a formal system with complex and not always consistent rules, it is also that case that, in most of human language use (and especially everyday use), it is about a world external to the speaker/writer and audience/reader. It also seem to me that, given the way LLMs are constructed and trained, they gain a good grasp of the formal properties of language (meaning the statistical properties of language as it is actually used, not just its grammar.) Does it also grasp the about-the-world aspect of language? I don't think we rationally have to assume the former leads to the latter, and we should be skeptical, firstly because we know how LLMs are made and secondly our prior experiences with language use has always been with people, for whom the relationship between language and the real world is so clear and ubiquitous that it is hard for us to consider these aspects separately, possibly leading us to anthropomorphize LLMs. If they do grasp the about-the-world aspect of language, then it seems to me just a short, and perhaps inevitable, step from there to self-awareness - seeing oneself as an individual in that world - and that would be a very big deal, IMHO.
        zeroCalories 3y ago
        When I say doubtful, I mean in a practical sense. If you view the world as a collection of facts, describing even mundane things fully requires an immense number of facts. You require an abstraction, and it's doubtful that a useful abstraction won't wash away details about the world. I also think you're making an arbitrary distinction between human input data, and LLM input data. I could ask the exact same questions about a human's understanding of the real world. All I can say about both humans and LLMs is that they seem to both make good predictions.
- causal 3y ago
  This really misrepresents what is happening in these models. Complex semantics are represented in the latent space. Relationships between embeddings have meanings we're able to sample to get these probabilistic strings - the fact that probabilities are involved does not mean there is no understanding behind the output.
  - mannykannot 3y ago
    > Complex semantics are represented in the latent space. In this particular case, what's going on doesn't seem all that complex. > The fact that probabilities are involved does not mean there is no understanding behind the output. And that, in turn, does not mean that there is - which is the question of interest here. The burden of proof is on anyone claiming to know the answer, either way.

mannykannot 3y ago

It is likely it has seen similar question-answer pairs, and it seems entirely plausible that substituting 2024 where a different year appeared scores highly on the most-probable-next-token measure, based only on those similar examples. The same caveat as on my previous reply applies here.

zeroCalories 3y ago
That's fine. I'm simply stating that an inability of one particular(locally ran 7b) LLM to answer a question does not disprove the intelligence of LLMs. But to answer your point, if the statistical model is complex enough, is that not intelligence? We could keep playing this game, and at some point the LLM might mess up, but I imagine we can get pretty far with the best models available.
- mannykannot 3y ago
  > if the statistical model is complex enough, is that not intelligence? Well, that is the question. I have speculated a bit more about that in these threads and elsewhere. One of the issues is whether a vast knowledge of the statistics drawn from language use can lead to an understanding that the language is about an external world. https://news.ycombinator.com/item?id=39501090 https://news.ycombinator.com/item?id=39501090
  - zeroCalories 3y ago
    It's doubtful that any language can fully express the medium in which it is expressed. But that shouldn't discourage us about an LLM's intelligence, as the sense data you get about the world is, in a sense, also a language. An LLM has a different view of the world and reality than us, but it still has a view.
    - mannykannot 3y ago
      > It's doubtful that any language can fully express the medium in which it is expressed. I feel I am missing the point here, maybe because I see language as being substrate-independent: while it needs some medium in which to be expressed, any medium which allows for binary distinctions is sufficient. The medium seems to me to be an issue independent of the fact that while language is a formal system with complex and not always consistent rules, it is also that case that, in most of human language use (and especially everyday use), it is about a world external to the speaker/writer and audience/reader. It also seem to me that, given the way LLMs are constructed and trained, they gain a good grasp of the formal properties of language (meaning the statistical properties of language as it is actually used, not just its grammar.) Does it also grasp the about-the-world aspect of language? I don't think we rationally have to assume the former leads to the latter, and we should be skeptical, firstly because we know how LLMs are made and secondly our prior experiences with language use has always been with people, for whom the relationship between language and the real world is so clear and ubiquitous that it is hard for us to consider these aspects separately, possibly leading us to anthropomorphize LLMs. If they do grasp the about-the-world aspect of language, then it seems to me just a short, and perhaps inevitable, step from there to self-awareness - seeing oneself as an individual in that world - and that would be a very big deal, IMHO.
      - zeroCalories 3y ago
        When I say doubtful, I mean in a practical sense. If you view the world as a collection of facts, describing even mundane things fully requires an immense number of facts. You require an abstraction, and it's doubtful that a useful abstraction won't wash away details about the world. I also think you're making an arbitrary distinction between human input data, and LLM input data. I could ask the exact same questions about a human's understanding of the real world. All I can say about both humans and LLMs is that they seem to both make good predictions.
causal 3y ago
This really misrepresents what is happening in these models. Complex semantics are represented in the latent space. Relationships between embeddings have meanings we're able to sample to get these probabilistic strings - the fact that probabilities are involved does not mean there is no understanding behind the output.
- mannykannot 3y ago
  > Complex semantics are represented in the latent space. In this particular case, what's going on doesn't seem all that complex. > The fact that probabilities are involved does not mean there is no understanding behind the output. And that, in turn, does not mean that there is - which is the question of interest here. The burden of proof is on anyone claiming to know the answer, either way.