3 ms·

Is it possible to tell intelligence from a simple lookup-table based script by asking questions? I think so. If you ask one question, there's a chance it was i

Is it possible to tell intelligence from a simple lookup-table based script by asking questions? I think so. If you ask one question, there's a chance it was in a lookup table. If you ask multiple questions from an immense set of questions (like trillions of trillions of trillions..., sampled uniformly), and it answers all correctly, then it's either true intelligence or a lookup table which covers this whole immense space. (I'd argue there's no difference, as a process which makes this nearly-infinite table has to be intelligent.) Same with self awareness - you can ask questions where it applies...

_heimdall 2y ago
In either case you are only looking at inputs and outputs. The results may correlate with what you'd consider self-awareness or intelligence, but can you really say that's what you are set ng without understanding how the system works internally. LLMs are trained on a massive dataset and the resulting model is effectively a compressed representation of that dataset. On the surface you'd have no way of knowing whether the algorithm answering is in fact just a lookup table. This issue feels very similar to scientific modelling vs controlled studies. Modelling may show correlation, but it will never be able to show causation. Asking a system a bunch of questions and getting the right answers is the same, you're just coming up with a sample set of modelling data and attempting to interpret how the system likely worked only by looking at inputs and outputs.
- killerstorm 2y ago
  When I talk with a person I can tell that they are intelligent and self-aware. Claiming that self-awareness is unknowable concept is inherently unproductive. People have been using "theory of mind" in practice for millenia so we have to assume it's good for something, otherwise we won't go anywhere. I don't think that knowing internals is important - I don't reach for a scalpel to get what a person means.
  - _heimdall 2y ago
    When you're talking to a person, though, you also have an understanding of what a human is and what your own experience is. Its reasonable to interact with another human and expect that they are roughly similar to you, especially when your interactions match what you'd expect. That doesn't extend as well to other species, let along non-living things that are entirely different from us. They could seem intelligent from the outside but internally function like a lookup table. They also could externally seem like a lookup table while internally matching much better what wed consider intelligence. We don't have context of first hand experience that applies and we don't know what's going on inside the black box. With all that said, I'm phrasing this way more certain than I mean to. I wouldn't claim to know whether a box is intelligent or not, I'm just trying to point out how hard or impossible it would be today without knowing more about the box.
    - killerstorm 2y ago
      Well, I take this: https://x.com/GrantSlatton/status/1600388425651453953 https://x.com/GrantSlatton/status/1600388425651453953 as evidence that GPT-4 can understand Python, based on assumptions: 1. You cannot execute non-trivial programs without understanding computation/programming language 2. It's extremely unlikely that these kind of programs or outputs are available anywhere on the internet - so at very least GPT-4 was able to adapt extremely complex patterns in a way which nobody can comprehend 3. Nobody explicitly coded this, this capability have arisen from SGD-based training process
      - _heimdall 2y ago
        That's an interesting one, I'll have to think through that a bit more. Just first thoughts here, but I don't think (2) is off the table. The model wouldn't necessarily have to have been trained on the exact algorithm and outputs. Forcing the model to work a step at a time and show each step may push the model into a spot where it doesn't comprehend the entire algorithm but it has broken the work down to small enough steps that it looks similar enough to python code it was trained on that it can accurately predict the output. I'm also assuming here that the person posting it didn't try a number of times before GPT got it right, but they could have cherry picked. More importantly, though, we still have to assume this output would require python comprehension. We can't inspect the model as it works and don't know what is going on internally, it just appears to be a problem hard enough to require comprehension.

_heimdall 2y ago

In either case you are only looking at inputs and outputs. The results may correlate with what you'd consider self-awareness or intelligence, but can you really say that's what you are set ng without understanding how the system works internally. LLMs are trained on a massive dataset and the resulting model is effectively a compressed representation of that dataset. On the surface you'd have no way of knowing whether the algorithm answering is in fact just a lookup table. This issue feels very similar to scientific modelling vs controlled studies. Modelling may show correlation, but it will never be able to show causation. Asking a system a bunch of questions and getting the right answers is the same, you're just coming up with a sample set of modelling data and attempting to interpret how the system likely worked only by looking at inputs and outputs.

killerstorm 2y ago
When I talk with a person I can tell that they are intelligent and self-aware. Claiming that self-awareness is unknowable concept is inherently unproductive. People have been using "theory of mind" in practice for millenia so we have to assume it's good for something, otherwise we won't go anywhere. I don't think that knowing internals is important - I don't reach for a scalpel to get what a person means.
- _heimdall 2y ago
  When you're talking to a person, though, you also have an understanding of what a human is and what your own experience is. Its reasonable to interact with another human and expect that they are roughly similar to you, especially when your interactions match what you'd expect. That doesn't extend as well to other species, let along non-living things that are entirely different from us. They could seem intelligent from the outside but internally function like a lookup table. They also could externally seem like a lookup table while internally matching much better what wed consider intelligence. We don't have context of first hand experience that applies and we don't know what's going on inside the black box. With all that said, I'm phrasing this way more certain than I mean to. I wouldn't claim to know whether a box is intelligent or not, I'm just trying to point out how hard or impossible it would be today without knowing more about the box.
  - killerstorm 2y ago
    Well, I take this: https://x.com/GrantSlatton/status/1600388425651453953 https://x.com/GrantSlatton/status/1600388425651453953 as evidence that GPT-4 can understand Python, based on assumptions: 1. You cannot execute non-trivial programs without understanding computation/programming language 2. It's extremely unlikely that these kind of programs or outputs are available anywhere on the internet - so at very least GPT-4 was able to adapt extremely complex patterns in a way which nobody can comprehend 3. Nobody explicitly coded this, this capability have arisen from SGD-based training process
    - _heimdall 2y ago
      That's an interesting one, I'll have to think through that a bit more. Just first thoughts here, but I don't think (2) is off the table. The model wouldn't necessarily have to have been trained on the exact algorithm and outputs. Forcing the model to work a step at a time and show each step may push the model into a spot where it doesn't comprehend the entire algorithm but it has broken the work down to small enough steps that it looks similar enough to python code it was trained on that it can accurately predict the output. I'm also assuming here that the person posting it didn't try a number of times before GPT got it right, but they could have cherry picked. More importantly, though, we still have to assume this output would require python comprehension. We can't inspect the model as it works and don't know what is going on internally, it just appears to be a problem hard enough to require comprehension.