3 ms·

Real time post shifting and mental gymnastics is truly a sight to behold. No the output isn't perfect and should be double checked (as with any professional dom

Real time post shifting and mental gymnastics is truly a sight to behold. No the output isn't perfect and should be double checked (as with any professional domain - you think what an Engineer with decades of experience signs off hasn't been checked multiple times by multiple parties ?, you think a professional programmer writes code and..just deploys it ?) but LLMs reason just fine. If you have anything to say about that other than the nonsensical - "it's not true understanding just because", i'd love to hear it.

Madmallard 4y ago
This line of reasoning sort of goes both ways. Advocates and claimants of the technology arguing that humans reason no differently and that LLMs just given the right data will be better than everyone at everything related is just as hokey.
- famouswaffles 4y ago
  I didn't say anything about reasoning no differently. Suppose you have 2 equations. You don't know what these equations are. However, you know that for any input, the output is the same. Any mathematician worth his salt will tell you that given said information, those 2 equations are equal or equivalent. It doesn't actually matter what those equations are. Equation 1 could be say a+b and Equation 2 could be (a-5) + (b-5) + 10 Or more realistically, both equations could look vastly different. The point I'm driving home here is that true distinction reveals itself in results. The fallacy of the philosophical zombie is that there is this supposed important distinction between "true understanding" and "fake/mimicry/whatever understanding" and yet you can't actually test for it. You can't show this supposed huge difference. A distinction that can't be tested for is not a distinction.
  - underanalyzer 4y ago
    This analogy does not make sense to me. We do not have equivalence between all of the infinite inputs and outputs here, we have equivalence in a finite number of cases and known cases where the two functions (human output and llm output) diverge drastically. Any mathematician worth their salt would tell you these functions are definitely not equal. Now you could make the argument that these functions are close enough most of the time so it won't matter but unless you want to get really rigorous that's more of a stats / engineering perceptive not mathematics. Any more importantly that's very up for debate, especially in a high pressure situation like medicine. Of course these models are wild and I'm quite impressed with them. I still can be worried about the damage someone who doesn't think things through could cause by assuming GPT-4 has human or super human level intelligence in a niche, high impact field.
    - famouswaffles 4y ago
      The equation example was simply an illustration. The point I'm making is that I can quite clearly show output that demonstrates reasoning and understanding by any actual metric. That's not the problem. The problem is that when I do, the argument Quickly shifts to "it's not real understanding!". That is what is nonsensical here. It's actually nonsensical whatever domain you want to think about it in. Either your dog fetches what you throw at it or it doesn't. The idea of "pretend fetching" as any meaningful distinction is incredibly silly and makes no sense. If you want to tell me there's a special distinction cool but when you can't show me what that distinction is, how to test for it, the qualitative or quantitative differences then I'm going to throw your argument away because it's not a valid one. It's just an arbitrary line drawn on sand.
cscurmudgeon 4y ago
> but LLMs reason just fine. There is a lot of research that shows they have trouble reasoning. Folks working with LLMs and building them agree that they can't reason, yet pop-sci folks religiously insist they can reason. This is not even a controversial topic. https://arxiv.org/abs/2205.11502 https://arxiv.org/abs/2205.11502 Reminds me of this comic: https://www.smbc-comics.com/comic/2010-01-31 https://www.smbc-comics.com/comic/2010-01-31
- mgfist 4y ago
  I'm not the person you responded to, but that's an old paper. LLMs have come a long way since. Nothing conclusive but GPT4 does show some signs of something deeper. https://arxiv.org/pdf/2303.12712.pdf https://arxiv.org/pdf/2303.12712.pdf
  - cscurmudgeon 4y ago
    Yes, I have read that paper and have forwarded it eagerly to colleagues. Respectfully, please read the paper you have linked, especially section 8.2. Look at section 8.2 in the paper you linked: "8.2 Lack of planning in arithmetic/reasoning problems" > However, it seems that the autoregressive nature of the model which forces it to solve problems in a sequential fashion sometimes poses a more profound difficulty that cannot be remedied simply by instructing the model to find a step by step solution Nowhere in the paper is an evaluation on standard reasoning datasets (https://tptp.org https://tptp.org) because they are much much much tougher than the simple examples that GPT-4 struggles with. LLMs are a huge breakthrough but let us not muddy the science.
    - Sunhold 4y ago
      GPT 4 was evaluated on multiple reasoning benchmarks in the release paper.[1] The problems in TPTP are not really appropriate for a language model and exceed the capabilities of most humans working without tools. Clearly humans have some reasoning ability even if they cannot solve those problems. [1] https://openai.com/research/gpt-4 https://openai.com/research/gpt-4
- 4y ago

Madmallard 4y ago

This line of reasoning sort of goes both ways. Advocates and claimants of the technology arguing that humans reason no differently and that LLMs just given the right data will be better than everyone at everything related is just as hokey.

famouswaffles 4y ago
I didn't say anything about reasoning no differently. Suppose you have 2 equations. You don't know what these equations are. However, you know that for any input, the output is the same. Any mathematician worth his salt will tell you that given said information, those 2 equations are equal or equivalent. It doesn't actually matter what those equations are. Equation 1 could be say a+b and Equation 2 could be (a-5) + (b-5) + 10 Or more realistically, both equations could look vastly different. The point I'm driving home here is that true distinction reveals itself in results. The fallacy of the philosophical zombie is that there is this supposed important distinction between "true understanding" and "fake/mimicry/whatever understanding" and yet you can't actually test for it. You can't show this supposed huge difference. A distinction that can't be tested for is not a distinction.
- underanalyzer 4y ago
  This analogy does not make sense to me. We do not have equivalence between all of the infinite inputs and outputs here, we have equivalence in a finite number of cases and known cases where the two functions (human output and llm output) diverge drastically. Any mathematician worth their salt would tell you these functions are definitely not equal. Now you could make the argument that these functions are close enough most of the time so it won't matter but unless you want to get really rigorous that's more of a stats / engineering perceptive not mathematics. Any more importantly that's very up for debate, especially in a high pressure situation like medicine. Of course these models are wild and I'm quite impressed with them. I still can be worried about the damage someone who doesn't think things through could cause by assuming GPT-4 has human or super human level intelligence in a niche, high impact field.
  - famouswaffles 4y ago
    The equation example was simply an illustration. The point I'm making is that I can quite clearly show output that demonstrates reasoning and understanding by any actual metric. That's not the problem. The problem is that when I do, the argument Quickly shifts to "it's not real understanding!". That is what is nonsensical here. It's actually nonsensical whatever domain you want to think about it in. Either your dog fetches what you throw at it or it doesn't. The idea of "pretend fetching" as any meaningful distinction is incredibly silly and makes no sense. If you want to tell me there's a special distinction cool but when you can't show me what that distinction is, how to test for it, the qualitative or quantitative differences then I'm going to throw your argument away because it's not a valid one. It's just an arbitrary line drawn on sand.

cscurmudgeon 4y ago

> but LLMs reason just fine. There is a lot of research that shows they have trouble reasoning. Folks working with LLMs and building them agree that they can't reason, yet pop-sci folks religiously insist they can reason. This is not even a controversial topic. https://arxiv.org/abs/2205.11502 https://arxiv.org/abs/2205.11502 Reminds me of this comic: https://www.smbc-comics.com/comic/2010-01-31 https://www.smbc-comics.com/comic/2010-01-31

mgfist 4y ago
I'm not the person you responded to, but that's an old paper. LLMs have come a long way since. Nothing conclusive but GPT4 does show some signs of something deeper. https://arxiv.org/pdf/2303.12712.pdf https://arxiv.org/pdf/2303.12712.pdf
- cscurmudgeon 4y ago
  Yes, I have read that paper and have forwarded it eagerly to colleagues. Respectfully, please read the paper you have linked, especially section 8.2. Look at section 8.2 in the paper you linked: "8.2 Lack of planning in arithmetic/reasoning problems" > However, it seems that the autoregressive nature of the model which forces it to solve problems in a sequential fashion sometimes poses a more profound difficulty that cannot be remedied simply by instructing the model to find a step by step solution Nowhere in the paper is an evaluation on standard reasoning datasets (https://tptp.org https://tptp.org) because they are much much much tougher than the simple examples that GPT-4 struggles with. LLMs are a huge breakthrough but let us not muddy the science.
  - Sunhold 4y ago
    GPT 4 was evaluated on multiple reasoning benchmarks in the release paper.[1] The problems in TPTP are not really appropriate for a language model and exceed the capabilities of most humans working without tools. Clearly humans have some reasoning ability even if they cannot solve those problems. [1] https://openai.com/research/gpt-4 https://openai.com/research/gpt-4
4y ago