3 ms·

No it's not, it's a language model. It manipulates words. If you want reasoning, you should find the right tool. For example, encode your problem for an SMT sol

No it's not, it's a language model. It manipulates words. If you want reasoning, you should find the right tool. For example, encode your problem for an SMT solver (like Z3, NuSMV, etc). Let's say you have an elevator start on floor 11, and another at floor 3. The rules are, they can never occupy the same floor. Is it possible for the first elevator to ever reach floor 2? No, they would have to pass each other. GPT-4 may be able to come to the conclusion by accident. But not reliably.

canjobear 4y ago
So reasoning didn’t exist before SAT solvers?
- fallingmeat 4y ago
  sorry, that wasn't my point. I just meant, there are tools for that job. LLMs don't seem to be the most appropriate tool. Thanks for helping me to clarify
TarasBob 4y ago
The second elevator goes to floor 1, then the first goes to floor 2. Task accomplished, no?
- fallingmeat 4y ago
  Yes, if they were to move any number of floors at the same time step. But the language model isn't necessarily going to automatically discover those rules for you. Even if it did, the model won't necessarily be able to frame the problem as a BDD and solve it. At best (and the is a valuable trait) the LLM would be able to reframe your problem in such a way that you could discover the necessary precision in the problem statement. Perhaps it could even encode it in some syntax for another tool.
- fallingmeat 4y ago
  Just tried this with ChaGPT for giggles. It spit out words, but I wouldn't call it sound reasoning. __user__: - A system in composed of two elevators. - Each have their own unique velocity parameter. - At each time step in the system, the elevators can move up or down a number of floor equal to their velocity, or change their velocity. - Assume that both elevators move at the same time step. - The elevators may never occupy the same floor. - Elevator A starts on floor 11, and Elevator B at floor 3. - Both elevator velocities start at 1. Is it possible for A to ever reach floor 2? __assistant__: No, it is not possible for Elevator A to ever reach floor 2 while maintaining the rule that the two elevators may never occupy the same floor. To see why this is the case, note that Elevator A starts at floor 11 and Elevator B starts at floor 3. Since both elevators move at the same time step, Elevator A can never move to a floor lower than Elevator B without violating the rule that they may never occupy the same floor. Furthermore, since Elevator A and Elevator B have the same velocity at the start and Elevator B is already on a lower floor, Elevator B will always reach any floor below Elevator A before Elevator A can get there. This means that Elevator A can never occupy any floor that is below the current position of Elevator B, including floor 2. Therefore, it is not possible for Elevator A to ever reach floor 2 while maintaining the given constraints. __user__: what if at the first time step, both velocities became equal to 5? __assistant__: If both elevators have their velocities set to 5 at the first time step, then it is possible for Elevator A to reach floor 2. With both elevators moving at velocity 5, Elevator A will reach floor 6 after the first time step, while Elevator B will reach floor 8. Since Elevator A is now on a lower floor than Elevator B, it is possible for Elevator A to continue moving down to floor 2 without violating the constraint that the elevators may never occupy the same floor. Specifically, Elevator A can continue moving down at velocity 5, reaching floors 1 and 2 after two more time steps. Elevator B, with velocity 5, will reach floor 13 after the same two time steps. So, if both elevators have their velocities set to 5 at the first time step, then it is possible for Elevator A to reach floor 2 while maintaining the given constraints.

canjobear 4y ago

So reasoning didn’t exist before SAT solvers?

fallingmeat 4y ago
sorry, that wasn't my point. I just meant, there are tools for that job. LLMs don't seem to be the most appropriate tool. Thanks for helping me to clarify

TarasBob 4y ago

The second elevator goes to floor 1, then the first goes to floor 2. Task accomplished, no?

fallingmeat 4y ago
Yes, if they were to move any number of floors at the same time step. But the language model isn't necessarily going to automatically discover those rules for you. Even if it did, the model won't necessarily be able to frame the problem as a BDD and solve it. At best (and the is a valuable trait) the LLM would be able to reframe your problem in such a way that you could discover the necessary precision in the problem statement. Perhaps it could even encode it in some syntax for another tool.
fallingmeat 4y ago
Just tried this with ChaGPT for giggles. It spit out words, but I wouldn't call it sound reasoning. __user__: - A system in composed of two elevators. - Each have their own unique velocity parameter. - At each time step in the system, the elevators can move up or down a number of floor equal to their velocity, or change their velocity. - Assume that both elevators move at the same time step. - The elevators may never occupy the same floor. - Elevator A starts on floor 11, and Elevator B at floor 3. - Both elevator velocities start at 1. Is it possible for A to ever reach floor 2? __assistant__: No, it is not possible for Elevator A to ever reach floor 2 while maintaining the rule that the two elevators may never occupy the same floor. To see why this is the case, note that Elevator A starts at floor 11 and Elevator B starts at floor 3. Since both elevators move at the same time step, Elevator A can never move to a floor lower than Elevator B without violating the rule that they may never occupy the same floor. Furthermore, since Elevator A and Elevator B have the same velocity at the start and Elevator B is already on a lower floor, Elevator B will always reach any floor below Elevator A before Elevator A can get there. This means that Elevator A can never occupy any floor that is below the current position of Elevator B, including floor 2. Therefore, it is not possible for Elevator A to ever reach floor 2 while maintaining the given constraints. __user__: what if at the first time step, both velocities became equal to 5? __assistant__: If both elevators have their velocities set to 5 at the first time step, then it is possible for Elevator A to reach floor 2. With both elevators moving at velocity 5, Elevator A will reach floor 6 after the first time step, while Elevator B will reach floor 8. Since Elevator A is now on a lower floor than Elevator B, it is possible for Elevator A to continue moving down to floor 2 without violating the constraint that the elevators may never occupy the same floor. Specifically, Elevator A can continue moving down at velocity 5, reaching floors 1 and 2 after two more time steps. Elevator B, with velocity 5, will reach floor 13 after the same two time steps. So, if both elevators have their velocities set to 5 at the first time step, then it is possible for Elevator A to reach floor 2 while maintaining the given constraints.