6 ms·

Good thing it's "incapable of reasoning"!

by chpatrick 4y ago

jupp0r 4y ago
Is a normal chess program capable of reasoning?
- chpatrick 4y ago
  In the narrow field of chess, yes.
  - deleted 4y ago
    [deleted]
- wizzwizz4 4y ago
  I would say so. Logical reasoning is not a hard thing to program a computer to do: the really hard part is to reason about the real world, rather than just a manually-defined world model. Traditional algorithms are vanguards of pure reason. Neural networks are super-intuition. Stockfish unites the two, executing an efficient intuitive heuristic search over the solution space of “moves in a chess game” – but no matter how cool the robot arm you build for it, Stockfish could never learn to assemble furniture.
- Symmetry 4y ago
  The Monte Carlo analysis AlphaZero used functioned as a sort of multi-step reasoning for it. GPT can use its token buffer for some multi-step reasoning but that sort of interferes with providing a conversation with the user so it's much less effective.
Longlius 4y ago
It is incapable of reasoning, actually - at least in this case. It has no internal understanding of chess which is why it makes illegal moves.
- RyanCavanaugh 4y ago
  What experiment would you run to determine if a given text input / text output interface had an "internal understanding of chess"?
  - gwright 4y ago
    What if you prompted with something like: Let's play a game chess. Use the standard rules except that .... Basically perturb the context to something a human would easily adapt to if they first knew the rules of chess but that would be difficult (or at least not obvious) to extrapolate from training data by ChatGPT (or more generally an LLM)
  - jltsiren 4y ago
    I think internal understanding requires internal processing. According to this functional definition, the way we are currently using language models basically excludes understanding. We are asking them to dream up or brainstorm things – to tell us the first things they associate with the prompt. Maybe it's possible to set up the system with some kind of self-feedback loop, where it continues evaluating and improving its answers without further prompts. If that works, it would be one step closer to a true AGI that can be said to understand things. There is a lot of confusion around the Chinese Room Argument. I think it makes a valid point by demonstrating that input/output behavior alone is insufficient for evaluating whether a system is intelligent and understands things. In order to do that, we need to see (or assume) the internal mechanism.
    - chpatrick 4y ago
      > Maybe it's possible to set up the system with some kind of self-feedback loop, where it continues evaluating and improving its answers without further prompts. It can do that while it generates output. Humans do the same thing when they figure out what they really mean while they're trying to express it.
      - jltsiren 4y ago

Good thing it's "incapable of reasoning"!

jupp0r 4y ago

Is a normal chess program capable of reasoning?

chpatrick 4y ago
In the narrow field of chess, yes.
- deleted 4y ago
  [deleted]
wizzwizz4 4y ago
I would say so. Logical reasoning is not a hard thing to program a computer to do: the really hard part is to reason about the real world, rather than just a manually-defined world model. Traditional algorithms are vanguards of pure reason. Neural networks are super-intuition. Stockfish unites the two, executing an efficient intuitive heuristic search over the solution space of “moves in a chess game” – but no matter how cool the robot arm you build for it, Stockfish could never learn to assemble furniture.
Symmetry 4y ago
The Monte Carlo analysis AlphaZero used functioned as a sort of multi-step reasoning for it. GPT can use its token buffer for some multi-step reasoning but that sort of interferes with providing a conversation with the user so it's much less effective.

Longlius 4y ago

It is incapable of reasoning, actually - at least in this case. It has no internal understanding of chess which is why it makes illegal moves.

RyanCavanaugh 4y ago
What experiment would you run to determine if a given text input / text output interface had an "internal understanding of chess"?
- gwright 4y ago
  What if you prompted with something like: Let's play a game chess. Use the standard rules except that .... Basically perturb the context to something a human would easily adapt to if they first knew the rules of chess but that would be difficult (or at least not obvious) to extrapolate from training data by ChatGPT (or more generally an LLM)
- jltsiren 4y ago
  I think internal understanding requires internal processing. According to this functional definition, the way we are currently using language models basically excludes understanding. We are asking them to dream up or brainstorm things – to tell us the first things they associate with the prompt. Maybe it's possible to set up the system with some kind of self-feedback loop, where it continues evaluating and improving its answers without further prompts. If that works, it would be one step closer to a true AGI that can be said to understand things. There is a lot of confusion around the Chinese Room Argument. I think it makes a valid point by demonstrating that input/output behavior alone is insufficient for evaluating whether a system is intelligent and understands things. In order to do that, we need to see (or assume) the internal mechanism.
  - chpatrick 4y ago
    > Maybe it's possible to set up the system with some kind of self-feedback loop, where it continues evaluating and improving its answers without further prompts. It can do that while it generates output. Humans do the same thing when they figure out what they really mean while they're trying to express it.
    - jltsiren 4y ago