3 ms·

What test would you give it to differentiate mimicking from thinking? I would like to experiment and test that out. I have seen many claims that transformer mo

What test would you give it to differentiate mimicking from thinking? I would like to experiment and test that out. I have seen many claims that transformer models can't "think" but I have not seen any purposed tangible tests to verify or refute the claim.

fl7305 4y ago
I believe GPT-4 is able to perform some level of logical reasoning. Simpler models look very good on the surface, but there are some examples where this starts to break own. Example: "Is it legal for a man to marry his widow's sister?" The catch here is the man is dead, which is the reason he can't marry. GPT-4 gets this right. Simpler models however can be prompted to perform step-by-step reasoning where they figure out that dead men can't marry, and that a "man's widow" means he's dead, but the model still can't be coaxed into figuring out that he can't marry in the scenario above. I'm guessing GPT-4 still has similar hangups where it gets stuck and can't get out no matter the coaxing, although I don't have any examples handy. Anyone? This doesn't mean GPT-4 is a stochastic parrot, just that there are still flaws in its reasoning abilities.
- UncleEntity 4y ago
  I had to look it up… Turns out posthumous marriages are legal in some places so a French man can legally marry his widow’s sister assuming it doesn’t run afoul of other laws like (which I didn’t check the legality of) polygamy.
  - CamperBob2 4y ago
    ISTR this sort of thing is also kosher in the Mormon church.
imiric 4y ago
That question can't be answered reliably without knowing the training data used. LLMs are great at being interfaces to the data they were trained on. Ask them anything beyond that, and it becomes clear they're not capable of thinking like humans do, i.e. producing novel ideas. It's kind of absurd we're having this discussion, TBH. The only reason it's difficult to distinguish mimicry from thinking is because of the sheer volume of data we're talking about here, and some clever algorithms that find correlations in it, and can synthesize a human-like response. This is exacerbated by the fact that human education is built around memorization, and that we equate intelligence with being able to regurgitate previously known facts. So if this is your measuring stick, then by all means LLMs can be considered intelligent agents capable of rational thought, when in reality this is far from the truth.
- flangola7 4y ago
  It doesn't know anything post September 2021, that's plenty of data that can be used to test its reasoning abilities and yet it still succeeds. It doesn't know specific facts about world events, but it can clearly reason about them if you give it a brief blurb about what it has missed out on - just like a human could after waking from long coma. >the sheer volume of data we're talking about here, and some clever algorithms that find correlations in it, and can synthesize a human-like response. If you swap the volume of data from "terabytes of UTF-8" to "decades of continuous input from the 5 senses," this doesn't sound very different than what you or I do already. The fact that it's difficult to differentiate mimicking from true thinking strongly suggests that there really isn't a difference, or at minimum not one that has any real world practical significance. If there is a scenario where the difference does have practical significance, what scenario is that? Because I sure can't think of one.

fl7305 4y ago

I believe GPT-4 is able to perform some level of logical reasoning. Simpler models look very good on the surface, but there are some examples where this starts to break own. Example: "Is it legal for a man to marry his widow's sister?" The catch here is the man is dead, which is the reason he can't marry. GPT-4 gets this right. Simpler models however can be prompted to perform step-by-step reasoning where they figure out that dead men can't marry, and that a "man's widow" means he's dead, but the model still can't be coaxed into figuring out that he can't marry in the scenario above. I'm guessing GPT-4 still has similar hangups where it gets stuck and can't get out no matter the coaxing, although I don't have any examples handy. Anyone? This doesn't mean GPT-4 is a stochastic parrot, just that there are still flaws in its reasoning abilities.

UncleEntity 4y ago
I had to look it up… Turns out posthumous marriages are legal in some places so a French man can legally marry his widow’s sister assuming it doesn’t run afoul of other laws like (which I didn’t check the legality of) polygamy.
- CamperBob2 4y ago
  ISTR this sort of thing is also kosher in the Mormon church.

imiric 4y ago

That question can't be answered reliably without knowing the training data used. LLMs are great at being interfaces to the data they were trained on. Ask them anything beyond that, and it becomes clear they're not capable of thinking like humans do, i.e. producing novel ideas. It's kind of absurd we're having this discussion, TBH. The only reason it's difficult to distinguish mimicry from thinking is because of the sheer volume of data we're talking about here, and some clever algorithms that find correlations in it, and can synthesize a human-like response. This is exacerbated by the fact that human education is built around memorization, and that we equate intelligence with being able to regurgitate previously known facts. So if this is your measuring stick, then by all means LLMs can be considered intelligent agents capable of rational thought, when in reality this is far from the truth.

flangola7 4y ago
It doesn't know anything post September 2021, that's plenty of data that can be used to test its reasoning abilities and yet it still succeeds. It doesn't know specific facts about world events, but it can clearly reason about them if you give it a brief blurb about what it has missed out on - just like a human could after waking from long coma. >the sheer volume of data we're talking about here, and some clever algorithms that find correlations in it, and can synthesize a human-like response. If you swap the volume of data from "terabytes of UTF-8" to "decades of continuous input from the 5 senses," this doesn't sound very different than what you or I do already. The fact that it's difficult to differentiate mimicking from true thinking strongly suggests that there really isn't a difference, or at minimum not one that has any real world practical significance. If there is a scenario where the difference does have practical significance, what scenario is that? Because I sure can't think of one.