2 ms·

Well, you can make falsifiable prediction about whether an AI "understands" something at a deep or shallow level, though both these concepts and the predictions

Well, you can make falsifiable prediction about whether an AI "understands" something at a deep or shallow level, though both these concepts and the predictions themselves will be a bit fuzzy. As a concrete example, take the "wolf, goat and cabbage cross a river" puzzle. you can make several experiments which distinguish at which level an AI "understands" it. - Can it solve the problem at all? - Can it solve the problem if you translate it in a different language? - Can it solve the problem if you switch the names of the characters around but maintain the framing of "a boat crossing the river"? - Can it solve an equivalent problem with completely different wording where the solution is still logically equivalent? A model that can do 1 but not 2-3 is probably just pattern matching a sequence of words; it doesn't "understand" the problem. A model that can do 1 and 2 but not 3 or 4 is still pattern-matching the problem, but it's matching abstract concepts (like "the concept of a wolf" instead of just the token "wolf"). A model that can do 3 but not 4 is probably pattern-matching the general-shape of the problem, as in "mutually-incompatible characters being transported on a boat". A model that can do 4 is the real deal. (I think ChatGPT currently sits between 2 and 3)

famouswaffles 4y ago
Gpt-4 can do all four. Just keep in mind that it has human like failure modes. It can give you an answer that is just applying common but false assumption reasoning steps. However if you rewrite the question to avoid biasing common priors, it gets it. And( Or at least with Bing), if you tell it it's making a wrong assumption somehow (not necessarily what the wedding assumption is), it gets it.
skybrian 4y ago
Yes, there are helpful experiments along these lines, but you need to be careful drawing conclusions because the output is random and it's easy to fall for gambling fallacies. This isn't like debugging a deterministic program; it can take more data than you might expect. For example, let's take "can it solve it at all." How many attempts will you give it before you give up? How many different prompts will you try? If it hasn't solved it yet, there's always an argument that it could, given a better prompt. Also, you might see a problem get solved the first time, take a screenshot, and then assume it can solve the problem reliably when it can't, it was just lucky that one time. Similarly for your other questions. If it does work, and you change one thing and it fails, is your change the cause or was it random? You need to try it both ways multiple times. For everyday purposes this often doesn't matter. It's like asking a random person for directions. If it works, maybe you don't care if it's repeatable, because you're never going to ask for the same directions again.

famouswaffles 4y ago

Gpt-4 can do all four. Just keep in mind that it has human like failure modes. It can give you an answer that is just applying common but false assumption reasoning steps. However if you rewrite the question to avoid biasing common priors, it gets it. And( Or at least with Bing), if you tell it it's making a wrong assumption somehow (not necessarily what the wedding assumption is), it gets it.

skybrian 4y ago

Yes, there are helpful experiments along these lines, but you need to be careful drawing conclusions because the output is random and it's easy to fall for gambling fallacies. This isn't like debugging a deterministic program; it can take more data than you might expect. For example, let's take "can it solve it at all." How many attempts will you give it before you give up? How many different prompts will you try? If it hasn't solved it yet, there's always an argument that it could, given a better prompt. Also, you might see a problem get solved the first time, take a screenshot, and then assume it can solve the problem reliably when it can't, it was just lucky that one time. Similarly for your other questions. If it does work, and you change one thing and it fails, is your change the cause or was it random? You need to try it both ways multiple times. For everyday purposes this often doesn't matter. It's like asking a random person for directions. If it works, maybe you don't care if it's repeatable, because you're never going to ask for the same directions again.