4 ms·

How do you know for sure how human reasoning occurs, and that it does not use similar mechanisms? (edited second question to make it focused)

by epups 3y ago

lordnacho 3y ago
This is the real big question. We don't know how human reasoning works, but we are happy to identify, entirely based on external interaction, what is and isn't correct human reasoning. Then someone comes up with P(A|B) is not reasoning, which seems like an internal mechanism. How do we square that?
freejazz 3y ago
The old "I know you are, but what am I" defense
DonaldPShimoda 3y ago
We don't need to know how human reasoning occurs. If you ask an AI chatbot a question like "count the number of occurrences of 'l' in the word 'lollapalooza'" (and then follow up on its responses), it becomes obvious that it is not "reasoning"; it is simply following a sort-of script. Here is a summarized transcript of this exercise with ChatGPT: --- Me: Count the number of occurrences of the letter 'l' in the word 'lollapalooza'. AI: The letter 'l' appears four times in the word 'lollapalooza'. Me: Describe the process by which you arrived at this answer. AI: [Provided a description of counting characters and then highlighted the first 'l' in the word four separate times to justify its answer.] Me: You counted thief first occurrence four times. That is incorrect. AI: I apologize for the mistake in my previous response. You are absolutely right. Here's the correct count of occurrences of the letter 'l' in the word 'lollapalooza': AI: [Provided a list of 8 "occurrences" of 'l' in the word 'lollapalooza', including replacing the first 'o' with an 'l' ("occurrence" 3) and counting letters like 'p', 'z', and 'a'. --- So even though the correct answer was provided at first, the model was not able to faithfully produce responses that backed up how it arrived at that conclusion, and when pressed to double-check its work it simply made things up. Even a toddler would not have claimed 'z' and 'a' were instances of 'l'. AI chatbots do not reason. They produce text responses to prompts based on stochastic methods. Trying to conflate the issue by suggesting that "we don't know how humans reason, so how do we know AI bots don't reason" is, frankly, absurd. We can easily demonstrate that they are inconsistent and have no concept of what they are writing responses about, as shown above.
- cmrdporcupine 3y ago
  Exactly, and this is what shows me that many people didn't seriously read much of the paper they're commenting on. The question is not whether an AI can get logical questions right. The question is whether it used reasoning to do it. And, like it or not, we have a formal definition of reasoning and logic, and long expertise in analyzing how that works. And it so happens that the paper's author is both a PhD in computer science but also a masters in philosophy and worked on proof engineering and logical deduction systems before. So the bulk of the paper is not about "ha ha it got it wrong", it's about: how did you get that answer? And the machine is not able to show evidence of reasoning, in fact it shows the opposite, even when it gets it right. Reasoning is a verb. It's an interactive, dialectical, process. LLMs don't seem to do that. They model a problem based on the relational/linguistic structures within it and related materials, but do not reason about it.
  - epups 3y ago
    > And, like it or not, we have a formal definition of reasoning and logic, and long expertise in analyzing how that works. Well then I guess all the humans in this thread would come to the exact same conclusion, because according to some expectations we are perfectly consistent and capable of logical reasoning. How would you describe the process where if you add context to a prompt - ie, if you prod the AI in a certain direction - you can drastically influence its results? Is it not using this context as an "argument"? Sure, we know for a fact that all they are doing is exploring a certain corner of a high-dimensional space, and that prompting gets us closer to a desired spot where the right answers reside. If this is true, then at least some logic and reasoning is encoded in language. And if this is also true, then perhaps what humans are performing is a similar trick.
    - AstralStorm 3y ago
      If so, then why ChatGPT gets stuck in a loop of producing the same wrong answers? And sometimes repeatedly producing new wrong ones? Does it immediately forget the context? It is being told how it is wrong and where in no uncertain terms. Yet it goes back to the same mistake, or skips steps for its reasoning convenience.

How do you know for sure how human reasoning occurs, and that it does not use similar mechanisms? (edited second question to make it focused)

by epups 3y ago

How do you know for sure how human reasoning occurs, and that it does not use similar mechanisms? (edited second question to make it focused)

lordnacho 3y ago

This is the real big question. We don't know how human reasoning works, but we are happy to identify, entirely based on external interaction, what is and isn't correct human reasoning. Then someone comes up with P(A|B) is not reasoning, which seems like an internal mechanism. How do we square that?

freejazz 3y ago

The old "I know you are, but what am I" defense

DonaldPShimoda 3y ago

We don't need to know how human reasoning occurs. If you ask an AI chatbot a question like "count the number of occurrences of 'l' in the word 'lollapalooza'" (and then follow up on its responses), it becomes obvious that it is not "reasoning"; it is simply following a sort-of script. Here is a summarized transcript of this exercise with ChatGPT: --- Me: Count the number of occurrences of the letter 'l' in the word 'lollapalooza'. AI: The letter 'l' appears four times in the word 'lollapalooza'. Me: Describe the process by which you arrived at this answer. AI: [Provided a description of counting characters and then highlighted the first 'l' in the word four separate times to justify its answer.] Me: You counted thief first occurrence four times. That is incorrect. AI: I apologize for the mistake in my previous response. You are absolutely right. Here's the correct count of occurrences of the letter 'l' in the word 'lollapalooza': AI: [Provided a list of 8 "occurrences" of 'l' in the word 'lollapalooza', including replacing the first 'o' with an 'l' ("occurrence" 3) and counting letters like 'p', 'z', and 'a'. --- So even though the correct answer was provided at first, the model was not able to faithfully produce responses that backed up how it arrived at that conclusion, and when pressed to double-check its work it simply made things up. Even a toddler would not have claimed 'z' and 'a' were instances of 'l'. AI chatbots do not reason. They produce text responses to prompts based on stochastic methods. Trying to conflate the issue by suggesting that "we don't know how humans reason, so how do we know AI bots don't reason" is, frankly, absurd. We can easily demonstrate that they are inconsistent and have no concept of what they are writing responses about, as shown above.

cmrdporcupine 3y ago
Exactly, and this is what shows me that many people didn't seriously read much of the paper they're commenting on. The question is not whether an AI can get logical questions right. The question is whether it used reasoning to do it. And, like it or not, we have a formal definition of reasoning and logic, and long expertise in analyzing how that works. And it so happens that the paper's author is both a PhD in computer science but also a masters in philosophy and worked on proof engineering and logical deduction systems before. So the bulk of the paper is not about "ha ha it got it wrong", it's about: how did you get that answer? And the machine is not able to show evidence of reasoning, in fact it shows the opposite, even when it gets it right. Reasoning is a verb. It's an interactive, dialectical, process. LLMs don't seem to do that. They model a problem based on the relational/linguistic structures within it and related materials, but do not reason about it.
- epups 3y ago
  > And, like it or not, we have a formal definition of reasoning and logic, and long expertise in analyzing how that works. Well then I guess all the humans in this thread would come to the exact same conclusion, because according to some expectations we are perfectly consistent and capable of logical reasoning. How would you describe the process where if you add context to a prompt - ie, if you prod the AI in a certain direction - you can drastically influence its results? Is it not using this context as an "argument"? Sure, we know for a fact that all they are doing is exploring a certain corner of a high-dimensional space, and that prompting gets us closer to a desired spot where the right answers reside. If this is true, then at least some logic and reasoning is encoded in language. And if this is also true, then perhaps what humans are performing is a similar trick.
  - AstralStorm 3y ago
    If so, then why ChatGPT gets stuck in a loop of producing the same wrong answers? And sometimes repeatedly producing new wrong ones? Does it immediately forget the context? It is being told how it is wrong and where in no uncertain terms. Yet it goes back to the same mistake, or skips steps for its reasoning convenience.