4 ms·

I haven’t read the paper but let’s be honest, current GPT style AI does not “think” in the way we think of humans thinking, and so it’s much more likely it’s “f

by yladiz 3y ago

I haven’t read the paper but let’s be honest, current GPT style AI does not “think” in the way we think of humans thinking, and so it’s much more likely it’s “faking till it’s making it” and there were enough resources in its training model that let ChatGPT answer whatever questions were given in a convincing way. This feels like that recent post about ChatGPT being able to answer medical Reddit questions more empathetically, even though the accuracy of the responses wasn’t evaluated.

UncleEntity 3y ago
So you’re saying they are drawing on past experience…err, training data and making inferences based on that?
- yladiz 3y ago
  You’re implying it’s thinking about the questions, and it’s not. This is the point: there’s no way GPT has theory of mind because it doesn’t think. It can mimic well but it’s not thinking.
  - CamperBob2 3y ago
    Mimicry is enough to account for the output, but it doesn't explain how the model is so good at understanding even vague or poorly-considered prompts. "Autocompletion" isn't an adequate explanation when neither the prompt nor the output has ever existed before. Something else is going on... something that obviously has a lot in common with how our own minds work.
    - fl7305 3y ago
      I agree. Another way to come to this conclusion is that the GPT-4 model is on the order of 1000B parameters, or on the order of a terabyte of data. That's not a lot of different sentences when you consider the combinatorical explosion.
  - kelseyfrog 3y ago
    Are you of the mind that there's some test, where if computers passed it, they would be functionally thinking, or more that it's ontologically impossible for computers to think?
    - yladiz 3y ago
      I'm more in the former camp, and I would agree that if the computer passed this theoretical test, it could be considered thinking. We would also need to define what it means to think, since it's an ambiguous term; thinking could mean that you can explain your reasoning behind giving an answer (something ChatGPT can generally do), thinking could mean producing something novel based on your experiences/"training data" (something ChatGPT can't do), or it could mean something else, and I think if we define what it means to think, then we could probably create a test that can be used to consider something functionally thinking. Of course, this is getting into the realm of philosophy, and what does it mean to think, so there's not a simple answer here, but I'd generally say that thinking implies a degree of novel concept generation, which is why I wouldn't consider ChatGPT to think. I could also understand an argument saying that it's impossible for a computer to think, but I don't think I'd agree with it.
  - flangola7 3y ago
    What test would you give it to differentiate mimicking from thinking? I would like to experiment and test that out. I have seen many claims that transformer models can't "think" but I have not seen any purposed tangible tests to verify or refute the claim.
    - fl7305 3y ago
      I believe GPT-4 is able to perform some level of logical reasoning. Simpler models look very good on the surface, but there are some examples where this starts to break own. Example: "Is it legal for a man to marry his widow's sister?" The catch here is the man is dead, which is the reason he can't marry. GPT-4 gets this right. Simpler models however can be prompted to perform step-by-step reasoning where they figure out that dead men can't marry, and that a "man's widow" means he's dead, but the model still can't be coaxed into figuring out that he can't marry in the scenario above. I'm guessing GPT-4 still has similar hangups where it gets stuck and can't get out no matter the coaxing, although I don't have any examples handy. Anyone? This doesn't mean GPT-4 is a stochastic parrot, just that there are still flaws in its reasoning abilities.
      - UncleEntity 3y ago
        I had to look it up… Turns out posthumous marriages are legal in some places so a French man can legally marry his widow’s sister assuming it doesn’t run afoul of other laws like (which I didn’t check the legality of) polygamy.
        CamperBob2 3y ago
        ISTR this sort of thing is also kosher in the Mormon church.
    - imiric 3y ago
      That question can't be answered reliably without knowing the training data used. LLMs are great at being interfaces to the data they were trained on. Ask them anything beyond that, and it becomes clear they're not capable of thinking like humans do, i.e. producing novel ideas. It's kind of absurd we're having this discussion, TBH. The only reason it's difficult to distinguish mimicry from thinking is because of the sheer volume of data we're talking about here, and some clever algorithms that find correlations in it, and can synthesize a human-like response. This is exacerbated by the fact that human education is built around memorization, and that we equate intelligence with being able to regurgitate previously known facts. So if this is your measuring stick, then by all means LLMs can be considered intelligent agents capable of rational thought, when in reality this is far from the truth.
  - circuit10 3y ago
    The easiest way to perfectly mimic thinking is to… just think. If this so-called “fake” thinking is indistinguishable from “real” thinking, is there really a difference?
fl7305 3y ago
> current GPT style AI does not “think” in the way we think of humans thinking On the contrary, isn't "step by step" reasoning exactly what we tell students to do to become better at solving complicated problems?
famouswaffles 3y ago
There is no meaningful difference between "fake" thinking and "real" thinking. Especially when you arrive at this imaginary distinction with "It's fake because it has to be fake"
- yladiz 3y ago
  But how would you define fake and real thinking? One done by a human and a machine?
  - famouswaffles 3y ago
    Shouldn't I be asking you that ? You're the one who thinks such a distinction exists no ? If thinking/understanding/reasoning/whatever can be fake, it should be testable. It should be a conclusion that can be reached from results not baseless speculation. I mean, what kind of huge difference can't be tested for ? If you tell me "this is fake gold" then there are numerous ways to distinguish fake gold from it's real counterpart, mostly by testing its physical properties. If the results and properties from "fake" [insert] and "real" [insert] can't be distinguished then well you've just made up a distinction that doesn't meaningfully exist.
    - bheadmaster 3y ago
      > If thinking/understanding/reasoning/whatever can be fake, it should be testable I agree with everything else you said, but this is a point I disagree with. Just because something isn't testable, doesn't mean it doesn't exist. Theoretically, the state of facts could be such that 1) human minds can think (as in, have consciousness and perform the process that is known to us as "human thinking") and 2) machines cannot think (as in, they don't have consciousness, but are literally just mechanisms that appear to provide same output as humans for a certain set of inputs). I don't think it's true, in fact, I don't think consciousness itself is anything more than a convenient illusion, but I still must point out that "what is real must be measurable" is not a valid argument. A more precise formulation might be "if thinking can be fake, the only way we can prove it is if it's testable - otherwise it doesn't make sense to discuss it".

by yladiz 3y ago

UncleEntity 3y ago

So you’re saying they are drawing on past experience…err, training data and making inferences based on that?

yladiz 3y ago
You’re implying it’s thinking about the questions, and it’s not. This is the point: there’s no way GPT has theory of mind because it doesn’t think. It can mimic well but it’s not thinking.
- CamperBob2 3y ago
  Mimicry is enough to account for the output, but it doesn't explain how the model is so good at understanding even vague or poorly-considered prompts. "Autocompletion" isn't an adequate explanation when neither the prompt nor the output has ever existed before. Something else is going on... something that obviously has a lot in common with how our own minds work.
  - fl7305 3y ago
    I agree. Another way to come to this conclusion is that the GPT-4 model is on the order of 1000B parameters, or on the order of a terabyte of data. That's not a lot of different sentences when you consider the combinatorical explosion.
- kelseyfrog 3y ago
  Are you of the mind that there's some test, where if computers passed it, they would be functionally thinking, or more that it's ontologically impossible for computers to think?
  - yladiz 3y ago
    I'm more in the former camp, and I would agree that if the computer passed this theoretical test, it could be considered thinking. We would also need to define what it means to think, since it's an ambiguous term; thinking could mean that you can explain your reasoning behind giving an answer (something ChatGPT can generally do), thinking could mean producing something novel based on your experiences/"training data" (something ChatGPT can't do), or it could mean something else, and I think if we define what it means to think, then we could probably create a test that can be used to consider something functionally thinking. Of course, this is getting into the realm of philosophy, and what does it mean to think, so there's not a simple answer here, but I'd generally say that thinking implies a degree of novel concept generation, which is why I wouldn't consider ChatGPT to think. I could also understand an argument saying that it's impossible for a computer to think, but I don't think I'd agree with it.
- flangola7 3y ago
  What test would you give it to differentiate mimicking from thinking? I would like to experiment and test that out. I have seen many claims that transformer models can't "think" but I have not seen any purposed tangible tests to verify or refute the claim.
  - fl7305 3y ago
    I believe GPT-4 is able to perform some level of logical reasoning. Simpler models look very good on the surface, but there are some examples where this starts to break own. Example: "Is it legal for a man to marry his widow's sister?" The catch here is the man is dead, which is the reason he can't marry. GPT-4 gets this right. Simpler models however can be prompted to perform step-by-step reasoning where they figure out that dead men can't marry, and that a "man's widow" means he's dead, but the model still can't be coaxed into figuring out that he can't marry in the scenario above. I'm guessing GPT-4 still has similar hangups where it gets stuck and can't get out no matter the coaxing, although I don't have any examples handy. Anyone? This doesn't mean GPT-4 is a stochastic parrot, just that there are still flaws in its reasoning abilities.
    - UncleEntity 3y ago
      I had to look it up… Turns out posthumous marriages are legal in some places so a French man can legally marry his widow’s sister assuming it doesn’t run afoul of other laws like (which I didn’t check the legality of) polygamy.
      - CamperBob2 3y ago
        ISTR this sort of thing is also kosher in the Mormon church.
  - imiric 3y ago
    That question can't be answered reliably without knowing the training data used. LLMs are great at being interfaces to the data they were trained on. Ask them anything beyond that, and it becomes clear they're not capable of thinking like humans do, i.e. producing novel ideas. It's kind of absurd we're having this discussion, TBH. The only reason it's difficult to distinguish mimicry from thinking is because of the sheer volume of data we're talking about here, and some clever algorithms that find correlations in it, and can synthesize a human-like response. This is exacerbated by the fact that human education is built around memorization, and that we equate intelligence with being able to regurgitate previously known facts. So if this is your measuring stick, then by all means LLMs can be considered intelligent agents capable of rational thought, when in reality this is far from the truth.
- circuit10 3y ago
  The easiest way to perfectly mimic thinking is to… just think. If this so-called “fake” thinking is indistinguishable from “real” thinking, is there really a difference?

fl7305 3y ago

> current GPT style AI does not “think” in the way we think of humans thinking On the contrary, isn't "step by step" reasoning exactly what we tell students to do to become better at solving complicated problems?

famouswaffles 3y ago

There is no meaningful difference between "fake" thinking and "real" thinking. Especially when you arrive at this imaginary distinction with "It's fake because it has to be fake"

yladiz 3y ago
But how would you define fake and real thinking? One done by a human and a machine?
- famouswaffles 3y ago
  Shouldn't I be asking you that ? You're the one who thinks such a distinction exists no ? If thinking/understanding/reasoning/whatever can be fake, it should be testable. It should be a conclusion that can be reached from results not baseless speculation. I mean, what kind of huge difference can't be tested for ? If you tell me "this is fake gold" then there are numerous ways to distinguish fake gold from it's real counterpart, mostly by testing its physical properties. If the results and properties from "fake" [insert] and "real" [insert] can't be distinguished then well you've just made up a distinction that doesn't meaningfully exist.
  - bheadmaster 3y ago
    > If thinking/understanding/reasoning/whatever can be fake, it should be testable I agree with everything else you said, but this is a point I disagree with. Just because something isn't testable, doesn't mean it doesn't exist. Theoretically, the state of facts could be such that 1) human minds can think (as in, have consciousness and perform the process that is known to us as "human thinking") and 2) machines cannot think (as in, they don't have consciousness, but are literally just mechanisms that appear to provide same output as humans for a certain set of inputs). I don't think it's true, in fact, I don't think consciousness itself is anything more than a convenient illusion, but I still must point out that "what is real must be measurable" is not a valid argument. A more precise formulation might be "if thinking can be fake, the only way we can prove it is if it's testable - otherwise it doesn't make sense to discuss it".