3 ms·

> given I've seen ChatGPT provide reasoning at a level many humans fail to reach on a regular basis. and you don't know for sure if it was really reasoning or

> given I've seen ChatGPT provide reasoning at a level many humans fail to reach on a regular basis. and you don't know for sure if it was really reasoning or memorization and stochastic parroting. From another hand there are many research results demonstrating that neural networks have limitation in learning even something like multiplying numbers with many digits.

vidarh 3y ago
I don't know if humans are "really reasoning or memorization and stochastic parroting" either without a testable hypothesis of what makes something "really reasoning" as opposed to the other two. And most humans struggle with multiplying numbers with even quite few digits.
- riku_iki 3y ago
  > And most humans struggle with multiplying numbers with even quite few digits. most of the humans can learn the algorithm, and do many iterations reliably if you give pen and paper, current LLMs struggle with this. So, if we define reasoning as learning and executing algorithm of complexity N, than humans can learn more complex algorithms for higher N.
  - vidarh 3y ago
    Current LLMs have gotten better at this with each iteration without the extensive rote repetition of applying the algorithms following by additional training on corrections that we subject people to. Yet, still, I think you overestimate the average humans ability to reliably execute even algorithms that simple. I've had LLMs do that correctly up to far larger numbers than I've multiplied by hand. I could do it, but I'm not confident I wouldn't get sloppy and make mistakes without going back and reviewing my work. I've also had LLMs execute algorithms most people wouldn't even understand. E.g. converting between NFAs and DFAs, or taking a grammar in an unspecified BNF variant abd generate grammatically valid sentences in it, or conversely validating a sentence against the grammar of a fictional language. I think a whole lot of the arguments here assume a level of human reasoning well above average. That's not to say current LLMs do not also have glaring blind spots, but so it seems to me does people dismissing their apparent reasoning out of hand because of assumptions about human abilities we mostly haven't tested.
    - riku_iki 3y ago
      > I've had LLMs do that correctly up to far larger numbers than I've multiplied by hand. but given enough time you can multiply any numbers, while LLM has fundamental inability to learn algorithm. > I've also had LLMs execute algorithms most people wouldn't even understand. E.g. converting between NFAs and DFAs this is a not good quality discussion, such claims should be expressed in the form of some research results: describing what you do exactly, how data has been obtained, verified for correctness, what is complexity of your NFAs etc, otherwise I personally have hard time to judge how indicative your claim.
      - speedgoose 3y ago
        ChatGPT knows how to write Python code, run it, and use the results. It can also use WolframAlpha. It’s not a pure LLM solution but it’s good enough to multiply numbers. Have you tried it?
        vidarh 3y ago
        ChatGPT knows how to multiply step by step without either of those, but like a human, who'll be annoyed and usually sloppy if you ask them to do things step by step instead of taking shortcuts, you increasingly need to push it to get it to do so with care. The irony of these objections is that people speak out of a "but we learned to do that at school" way of thinking, often on the basis of above average skills, rather than considering that while most people maybe could (I'm not convinced), most people won't do it step by step unless you really goad them into it and pay attention that they don't skip or gloss over steps.
        riku_iki 3y ago
        The question is inability of ChatGPT to do logical reasoning. Multiplying numbers is a simple test to prove this. The same with coding: it can produce code for tasks where something similar was in training dataset, but if you ask to do completely unknown problem - it will fail.
      - vidarh 3y ago
        > but given enough time you can multiply any numbers, while LLM has fundamental inability to learn algorithm. GPT4 at least knows the algorithms - many alternative version - and will follow them. What it won't do, just like most humans, is take the time to go over it and review each step for possible errors without further prompting and inducement. Yes, humans could. Most humans don't, without outside pressure. With pressure applied and LLM will too. Again, this is an example of how people overestimate how humans do in the same condition. I would suggest you look up some studies on numeracy - sure, most people can learn if put through enough pressure, and good humans would do well against currebt LLMs, but average adult human numeracy is shockingly poor. Consider if you would question people's ability to reason in general if you find a few examples of humans who do poorly at it. It is fascinating when people compare LLMs this way against some platonic ideal of a human without considering that many of the failures of LLMs to stick to the prompt occurs in humans too. E.g. when you argue a human could do something with enough time, you leave out the effort you'd need to expend to convince said human to do so, and follow your instructions carefully enough. Yes, there are still plenty of conditions where humans can beat LLMs given sufficient inducement, and even in many cases easily. But that is a very different subject. > this is a not good quality discussion, such claims should be expressed in the form of some research results: describing what you do exactly, how data has been obtained, verified for correctness, what is complexity of your NFAs etc, otherwise I personally have hard time to judge how indicative your claim I'll happily do that if you do the same for your unsubstantiated claims first. Otherwise you can simply actually try to ask ChatGPT about these things yourself, and then try to ask a random person about them.
- LabMechanic 3y ago
  Okay, I mentioned this in some of my past comments, but try this: Ask your "AI" something about a non-popular domain (e.g., Clifford algebra) or even ask it to derive something (e.g., the geometric product) given some axioms and definitions. Mathematics is precise as it gets, yet it failed to derive the geometric product given a set of axioms (e.g., distributive property) and definitions. Ask it to derive a theorem from perspective geometry. Or, try something else that is not so popular or widespread (something outside of math), I suspect, that it also does a poor job deriving things from principles (axioms, definitions, theorems).
bhdlr 3y ago
Is there a difference between memorization + stochastic parroting if it is indistinguishable from human understanding? If a hunk of metal looks like a car, sounds like a car, and drives like a car is it a car?
- riku_iki 3y ago
  it is indistinguishable on some trivial results, but not scalable for next level of complexity.
- LabMechanic 3y ago
  Who knows, but until it can fool me by randomness (like Nassim Taleb suggests), i.e., deriving something given a set of axioms and definitions (from a non-widespread domain), then I might join your "ignorance is bliss" argument. Also, hype is good for the stock market, so keep hyping it up? At the end, Elon Musk success seems to stem from hype and sensationalism. CNN (i.e., "breaking news" everywhere) has financial success due to sensationalism and hype, so hype is good, I guess?
LabMechanic 3y ago
Yep, or somehow they make it apparent that they indeed "parrot" stuff they had in their large dataset. See my past comments for details. Fooled by randomness? (See Nassim Nicholas Taleb)

vidarh 3y ago

I don't know if humans are "really reasoning or memorization and stochastic parroting" either without a testable hypothesis of what makes something "really reasoning" as opposed to the other two. And most humans struggle with multiplying numbers with even quite few digits.

riku_iki 3y ago
> And most humans struggle with multiplying numbers with even quite few digits. most of the humans can learn the algorithm, and do many iterations reliably if you give pen and paper, current LLMs struggle with this. So, if we define reasoning as learning and executing algorithm of complexity N, than humans can learn more complex algorithms for higher N.
- vidarh 3y ago
  Current LLMs have gotten better at this with each iteration without the extensive rote repetition of applying the algorithms following by additional training on corrections that we subject people to. Yet, still, I think you overestimate the average humans ability to reliably execute even algorithms that simple. I've had LLMs do that correctly up to far larger numbers than I've multiplied by hand. I could do it, but I'm not confident I wouldn't get sloppy and make mistakes without going back and reviewing my work. I've also had LLMs execute algorithms most people wouldn't even understand. E.g. converting between NFAs and DFAs, or taking a grammar in an unspecified BNF variant abd generate grammatically valid sentences in it, or conversely validating a sentence against the grammar of a fictional language. I think a whole lot of the arguments here assume a level of human reasoning well above average. That's not to say current LLMs do not also have glaring blind spots, but so it seems to me does people dismissing their apparent reasoning out of hand because of assumptions about human abilities we mostly haven't tested.
  - riku_iki 3y ago
    > I've had LLMs do that correctly up to far larger numbers than I've multiplied by hand. but given enough time you can multiply any numbers, while LLM has fundamental inability to learn algorithm. > I've also had LLMs execute algorithms most people wouldn't even understand. E.g. converting between NFAs and DFAs this is a not good quality discussion, such claims should be expressed in the form of some research results: describing what you do exactly, how data has been obtained, verified for correctness, what is complexity of your NFAs etc, otherwise I personally have hard time to judge how indicative your claim.
    - speedgoose 3y ago
      ChatGPT knows how to write Python code, run it, and use the results. It can also use WolframAlpha. It’s not a pure LLM solution but it’s good enough to multiply numbers. Have you tried it?
      - vidarh 3y ago
        ChatGPT knows how to multiply step by step without either of those, but like a human, who'll be annoyed and usually sloppy if you ask them to do things step by step instead of taking shortcuts, you increasingly need to push it to get it to do so with care. The irony of these objections is that people speak out of a "but we learned to do that at school" way of thinking, often on the basis of above average skills, rather than considering that while most people maybe could (I'm not convinced), most people won't do it step by step unless you really goad them into it and pay attention that they don't skip or gloss over steps.
      - riku_iki 3y ago
        The question is inability of ChatGPT to do logical reasoning. Multiplying numbers is a simple test to prove this. The same with coding: it can produce code for tasks where something similar was in training dataset, but if you ask to do completely unknown problem - it will fail.
    - vidarh 3y ago
      > but given enough time you can multiply any numbers, while LLM has fundamental inability to learn algorithm. GPT4 at least knows the algorithms - many alternative version - and will follow them. What it won't do, just like most humans, is take the time to go over it and review each step for possible errors without further prompting and inducement. Yes, humans could. Most humans don't, without outside pressure. With pressure applied and LLM will too. Again, this is an example of how people overestimate how humans do in the same condition. I would suggest you look up some studies on numeracy - sure, most people can learn if put through enough pressure, and good humans would do well against currebt LLMs, but average adult human numeracy is shockingly poor. Consider if you would question people's ability to reason in general if you find a few examples of humans who do poorly at it. It is fascinating when people compare LLMs this way against some platonic ideal of a human without considering that many of the failures of LLMs to stick to the prompt occurs in humans too. E.g. when you argue a human could do something with enough time, you leave out the effort you'd need to expend to convince said human to do so, and follow your instructions carefully enough. Yes, there are still plenty of conditions where humans can beat LLMs given sufficient inducement, and even in many cases easily. But that is a very different subject. > this is a not good quality discussion, such claims should be expressed in the form of some research results: describing what you do exactly, how data has been obtained, verified for correctness, what is complexity of your NFAs etc, otherwise I personally have hard time to judge how indicative your claim I'll happily do that if you do the same for your unsubstantiated claims first. Otherwise you can simply actually try to ask ChatGPT about these things yourself, and then try to ask a random person about them.
LabMechanic 3y ago
Okay, I mentioned this in some of my past comments, but try this: Ask your "AI" something about a non-popular domain (e.g., Clifford algebra) or even ask it to derive something (e.g., the geometric product) given some axioms and definitions. Mathematics is precise as it gets, yet it failed to derive the geometric product given a set of axioms (e.g., distributive property) and definitions. Ask it to derive a theorem from perspective geometry. Or, try something else that is not so popular or widespread (something outside of math), I suspect, that it also does a poor job deriving things from principles (axioms, definitions, theorems).

bhdlr 3y ago

Is there a difference between memorization + stochastic parroting if it is indistinguishable from human understanding? If a hunk of metal looks like a car, sounds like a car, and drives like a car is it a car?

riku_iki 3y ago
it is indistinguishable on some trivial results, but not scalable for next level of complexity.
LabMechanic 3y ago
Who knows, but until it can fool me by randomness (like Nassim Taleb suggests), i.e., deriving something given a set of axioms and definitions (from a non-widespread domain), then I might join your "ignorance is bliss" argument. Also, hype is good for the stock market, so keep hyping it up? At the end, Elon Musk success seems to stem from hype and sensationalism. CNN (i.e., "breaking news" everywhere) has financial success due to sensationalism and hype, so hype is good, I guess?

LabMechanic 3y ago

Yep, or somehow they make it apparent that they indeed "parrot" stuff they had in their large dataset. See my past comments for details. Fooled by randomness? (See Nassim Nicholas Taleb)