3 ms·

Arguing over whether it “understands” or not is bad philosophy. It’s like there’s a magic show and you’re arguing over whether it’s “real magic” or whether the

Arguing over whether it “understands” or not is bad philosophy. It’s like there’s a magic show and you’re arguing over whether it’s “real magic” or whether there’s “some trick to it.” There are always tricks, but until you know what they are, the mystery is still there and you haven’t solved it. If God told you “yes it understands” or “no it doesn’t,” what would you have learned? The mystery would still be there. It’s like the Douglas Adams story about the machine that answered 42. We know the basic architecture of large language models, but hardly anything about how they calculate anything specific. That’s the mystery. It will take research, not casual tinkering. Screenshots show how it reacted one time, but the output is random, so you need to regenerate a lot to get a sense of the distribution. Such experiments will help, but I suspect really figuring it out will require some good debugging tools.

PoignardAzur 4y ago
Well, you can make falsifiable prediction about whether an AI "understands" something at a deep or shallow level, though both these concepts and the predictions themselves will be a bit fuzzy. As a concrete example, take the "wolf, goat and cabbage cross a river" puzzle. you can make several experiments which distinguish at which level an AI "understands" it. - Can it solve the problem at all? - Can it solve the problem if you translate it in a different language? - Can it solve the problem if you switch the names of the characters around but maintain the framing of "a boat crossing the river"? - Can it solve an equivalent problem with completely different wording where the solution is still logically equivalent? A model that can do 1 but not 2-3 is probably just pattern matching a sequence of words; it doesn't "understand" the problem. A model that can do 1 and 2 but not 3 or 4 is still pattern-matching the problem, but it's matching abstract concepts (like "the concept of a wolf" instead of just the token "wolf"). A model that can do 3 but not 4 is probably pattern-matching the general-shape of the problem, as in "mutually-incompatible characters being transported on a boat". A model that can do 4 is the real deal. (I think ChatGPT currently sits between 2 and 3)
- famouswaffles 4y ago
  Gpt-4 can do all four. Just keep in mind that it has human like failure modes. It can give you an answer that is just applying common but false assumption reasoning steps. However if you rewrite the question to avoid biasing common priors, it gets it. And( Or at least with Bing), if you tell it it's making a wrong assumption somehow (not necessarily what the wedding assumption is), it gets it.
- skybrian 4y ago
  Yes, there are helpful experiments along these lines, but you need to be careful drawing conclusions because the output is random and it's easy to fall for gambling fallacies. This isn't like debugging a deterministic program; it can take more data than you might expect. For example, let's take "can it solve it at all." How many attempts will you give it before you give up? How many different prompts will you try? If it hasn't solved it yet, there's always an argument that it could, given a better prompt. Also, you might see a problem get solved the first time, take a screenshot, and then assume it can solve the problem reliably when it can't, it was just lucky that one time. Similarly for your other questions. If it does work, and you change one thing and it fails, is your change the cause or was it random? You need to try it both ways multiple times. For everyday purposes this often doesn't matter. It's like asking a random person for directions. If it works, maybe you don't care if it's repeatable, because you're never going to ask for the same directions again.

PoignardAzur 4y ago

Well, you can make falsifiable prediction about whether an AI "understands" something at a deep or shallow level, though both these concepts and the predictions themselves will be a bit fuzzy. As a concrete example, take the "wolf, goat and cabbage cross a river" puzzle. you can make several experiments which distinguish at which level an AI "understands" it. - Can it solve the problem at all? - Can it solve the problem if you translate it in a different language? - Can it solve the problem if you switch the names of the characters around but maintain the framing of "a boat crossing the river"? - Can it solve an equivalent problem with completely different wording where the solution is still logically equivalent? A model that can do 1 but not 2-3 is probably just pattern matching a sequence of words; it doesn't "understand" the problem. A model that can do 1 and 2 but not 3 or 4 is still pattern-matching the problem, but it's matching abstract concepts (like "the concept of a wolf" instead of just the token "wolf"). A model that can do 3 but not 4 is probably pattern-matching the general-shape of the problem, as in "mutually-incompatible characters being transported on a boat". A model that can do 4 is the real deal. (I think ChatGPT currently sits between 2 and 3)

famouswaffles 4y ago
Gpt-4 can do all four. Just keep in mind that it has human like failure modes. It can give you an answer that is just applying common but false assumption reasoning steps. However if you rewrite the question to avoid biasing common priors, it gets it. And( Or at least with Bing), if you tell it it's making a wrong assumption somehow (not necessarily what the wedding assumption is), it gets it.
skybrian 4y ago
Yes, there are helpful experiments along these lines, but you need to be careful drawing conclusions because the output is random and it's easy to fall for gambling fallacies. This isn't like debugging a deterministic program; it can take more data than you might expect. For example, let's take "can it solve it at all." How many attempts will you give it before you give up? How many different prompts will you try? If it hasn't solved it yet, there's always an argument that it could, given a better prompt. Also, you might see a problem get solved the first time, take a screenshot, and then assume it can solve the problem reliably when it can't, it was just lucky that one time. Similarly for your other questions. If it does work, and you change one thing and it fails, is your change the cause or was it random? You need to try it both ways multiple times. For everyday purposes this often doesn't matter. It's like asking a random person for directions. If it works, maybe you don't care if it's repeatable, because you're never going to ask for the same directions again.