5 ms·

Have you ever actually bothered testing your assumption that chatgpt can't reason. And I don't mean that it fails to reason properly about certain questions, t

Have you ever actually bothered testing your assumption that chatgpt can't reason. And I don't mean that it fails to reason properly about certain questions, that's trivially easy to show, but is less interesting than many people think it is, because humans can't reason properly about many questions (see, for example, the Monty Hall problem). Test your assumption that it can never reason properly about any question not in its data set. It's way easier, IME, to find examples where it does give the correct answer to novel problems than it is to trip it up with a difficult problem.

mjburgess 2y ago
It's data set is, approximately, everything ever written. We have no access to it. And this is widely studied. You can find trivial mistakes in apparent reasoning. There are two hypotheses: H1, the structure of a response from any given prompt is computed using distributional properties of historical data; H2: the response is computed via deduction from premises to conclusions of agent employing the semantics of the terms, their logical connections, and connections of relevance. In many cases a prompt/reply will confirm both hypotheses, hence confirmation bias and why we dont bother "confirming" any hypothesis. Rather to choose betweeen them, if you wanted to use data, you just find cases where reasoning fails in such a way that H1 is the more plausible answer. Such cases are easy to find, and across the literature. This whole thing is pointless however, because it's blindingly obvious from what a statistical AI algorithm does, which is empirical function fiting on historical datasets to approximate conditional probability distributions. This process is extremely well-understood, and we know necessarily that it is just approximating a historical distribution.
- stevenhuang 2y ago
  > This process is extremely well-understood, and we know necessarily that it is just approximating a historical distribution. Prove that this precludes formation of mind. Prove that human minds do not work this way. Until such premises are established, your argument is not sound. This leaves open the possibility what we're seeing is in fact a degree of reasoning ability in LLMs.
  - mjburgess 2y ago
    As I said in my other comment. Statistical models generate output with constant time/energy. So for any input y, and any output x, the computation is constant time. eg., generating any 100 tokens of output takes the same time as any other. If you reflect on that carefully, you'll note the only way this is possible is if the algorithm does not use any semantic features of the input. Indeed, if it has no relevant capacities at all. Consider e.g., the prompt, "imagine a world where..., and then infer..., and then what would be... ?" for various "..." we can choose, the output ought take arbitrarily longer to compute; but its constant. For any word: imagine, reason, suppose, simulate, believe, remember... denoting any alleged mental operation, we can trivially construct cases where actually performing this operation would take more or less time. "Given premises A, B, C, D,...; infer conclusion..." has a prompt reply which is constant in time regardless of how many premises we input. "Recall memories A, B,C,.." likewise. The only way that answering "yes" to a question, say, regardless of what that question is, taking the same time/energy/etc. to answer.. would be if no semantic feature of that question was being used in its answering. This is an LLM, it's all statistical AI: the number of operations performed per prediction is constant for all predictions. Almost no alleged capacities are consistent with this fact.
    - FeepingCreature 2y ago
      The human brain computes precisely the same amount of information every millisecond. That is, the computational power of a single neuron is necessarily capped by its switching speed. This proves humans cannot think.
      - mjburgess 2y ago
        Whether that is true or not, and it obviously isn't -- the energy used by the brain is highly variable -- that isnt the point. The point is that if I ask you to do some complex task that requires a one-word answer, you will take a long time to think about it. If I ask an LLM, it will always take the same amount of time, regardless of the task. It follows from P =/= NP that this should necessarily not happen if the LLM is actually evaluating the semantics of the question. However, it also follows from just the properties of what capacities are being claimed Reasoning, as an algorithm, is at least, O(NumberOfPremises) etc. The number of operations performed, in every case, for all statistical AI algs, is the same. It's not O(ANY PROPERTY OF THE PROBLEM) -- this is a catastrophe for anyone who would claim that a statistical algorithm considers the problem at all. Statistics here is just a very naive short-cut. Rather than solve the problem, you're just sampling from previous answers. This, of course, should be obvious to any one with the bare minimum of knowledge of applied statistics and the formalism of ML. Neverthless, of course, it isnt. One has to imagine that overwhelming amounts of corporate propaganda and scifi wishfulfilment are at work.
stevenhuang 2y ago
The guy you're responding to can be seen routinely interjecting with these kinds of unfounded assertions that LLMs can't reason everywhere LLMs are discussed. He can't help himself it seems. You'd think he'd engage with the responses he receives, maybe down-weight his convictions through exposure to reasonable arguments, but nope. An LLM would have the wherewithal to consider the possibility they're wrong, or to be agnostic in their beliefs until we ourselves understand better what it means to reason.

mjburgess 2y ago

It's data set is, approximately, everything ever written. We have no access to it. And this is widely studied. You can find trivial mistakes in apparent reasoning. There are two hypotheses: H1, the structure of a response from any given prompt is computed using distributional properties of historical data; H2: the response is computed via deduction from premises to conclusions of agent employing the semantics of the terms, their logical connections, and connections of relevance. In many cases a prompt/reply will confirm both hypotheses, hence confirmation bias and why we dont bother "confirming" any hypothesis. Rather to choose betweeen them, if you wanted to use data, you just find cases where reasoning fails in such a way that H1 is the more plausible answer. Such cases are easy to find, and across the literature. This whole thing is pointless however, because it's blindingly obvious from what a statistical AI algorithm does, which is empirical function fiting on historical datasets to approximate conditional probability distributions. This process is extremely well-understood, and we know necessarily that it is just approximating a historical distribution.

stevenhuang 2y ago
> This process is extremely well-understood, and we know necessarily that it is just approximating a historical distribution. Prove that this precludes formation of mind. Prove that human minds do not work this way. Until such premises are established, your argument is not sound. This leaves open the possibility what we're seeing is in fact a degree of reasoning ability in LLMs.
- mjburgess 2y ago
  As I said in my other comment. Statistical models generate output with constant time/energy. So for any input y, and any output x, the computation is constant time. eg., generating any 100 tokens of output takes the same time as any other. If you reflect on that carefully, you'll note the only way this is possible is if the algorithm does not use any semantic features of the input. Indeed, if it has no relevant capacities at all. Consider e.g., the prompt, "imagine a world where..., and then infer..., and then what would be... ?" for various "..." we can choose, the output ought take arbitrarily longer to compute; but its constant. For any word: imagine, reason, suppose, simulate, believe, remember... denoting any alleged mental operation, we can trivially construct cases where actually performing this operation would take more or less time. "Given premises A, B, C, D,...; infer conclusion..." has a prompt reply which is constant in time regardless of how many premises we input. "Recall memories A, B,C,.." likewise. The only way that answering "yes" to a question, say, regardless of what that question is, taking the same time/energy/etc. to answer.. would be if no semantic feature of that question was being used in its answering. This is an LLM, it's all statistical AI: the number of operations performed per prediction is constant for all predictions. Almost no alleged capacities are consistent with this fact.
  - FeepingCreature 2y ago
    The human brain computes precisely the same amount of information every millisecond. That is, the computational power of a single neuron is necessarily capped by its switching speed. This proves humans cannot think.
    - mjburgess 2y ago
      Whether that is true or not, and it obviously isn't -- the energy used by the brain is highly variable -- that isnt the point. The point is that if I ask you to do some complex task that requires a one-word answer, you will take a long time to think about it. If I ask an LLM, it will always take the same amount of time, regardless of the task. It follows from P =/= NP that this should necessarily not happen if the LLM is actually evaluating the semantics of the question. However, it also follows from just the properties of what capacities are being claimed Reasoning, as an algorithm, is at least, O(NumberOfPremises) etc. The number of operations performed, in every case, for all statistical AI algs, is the same. It's not O(ANY PROPERTY OF THE PROBLEM) -- this is a catastrophe for anyone who would claim that a statistical algorithm considers the problem at all. Statistics here is just a very naive short-cut. Rather than solve the problem, you're just sampling from previous answers. This, of course, should be obvious to any one with the bare minimum of knowledge of applied statistics and the formalism of ML. Neverthless, of course, it isnt. One has to imagine that overwhelming amounts of corporate propaganda and scifi wishfulfilment are at work.

stevenhuang 2y ago

The guy you're responding to can be seen routinely interjecting with these kinds of unfounded assertions that LLMs can't reason everywhere LLMs are discussed. He can't help himself it seems. You'd think he'd engage with the responses he receives, maybe down-weight his convictions through exposure to reasonable arguments, but nope. An LLM would have the wherewithal to consider the possibility they're wrong, or to be agnostic in their beliefs until we ourselves understand better what it means to reason.