5 ms·

I think this misunderstands the reason why people say LLMs are "just" statistical text predictors and misunderstands LLMs in general. Here's where you're wrong

by bccdee 3y ago

I think this misunderstands the reason why people say LLMs are "just" statistical text predictors and misunderstands LLMs in general. Here's where you're wrong: > The real reason behind this bizarre answer is that it matches the form of a more elaborate puzzle where a similar indirect approach is necessary. GPT4 gets caught up in the tendency to match previous examples, and this is a genuine reflection of its cognitive history; it started out as a statistical text predictor. This is not the same as not understanding the question; this is a reflection of a faulty cognitive architecture that can be fixed relatively easily. This is not anything that can be easily fixed. This type of faulty problem-solving is a fundamental property of how LLMs work. It can be attenuated or avoided, but not repaired or removed. LLMs work by assembling language into plausibly-human-written sentences, and if you ask them to engage in problem-solving, they will often write their way to a correct solution, but examples like the 12L/6L jug question are revealing: The LLM doesn't actually reason. It imitates the appearance of reasoning, and does so extremely well, but that's not the same. It doesn't understand why you would pour one jug into another; it's been taught that pouring jugs into each other and arriving at the required number of litres are characteristics of the solutions of similar problems it's been trained on, and so it does all of these things. LLMs imitate human writing, and can, by proxy, reproduce human reason, but they have a tenuous and indirect grasp on it, and this drawback is built into their design.

oezi 3y ago
I see absolutely no reason (except cost) that LLM interfaces shouldn't utilize an internal dialog which is hidden from the user and which can be queried from the first layer of interaction and potentially vice versa. This would emulate much of what we consider human reasoning. And if it quacks like a duck it might be a duck...
- bccdee 3y ago
  Asking the LLM to be more verbose can bring contradictions to the surface and lead to more reliable answers, but it still doesn't fix the underlying problem. Even when you ask it to show its work, it can still contradict itself, blindly imitate patterns in its training data, and contrive plausible-sounding but false answers. I've got a (rather long) transcript I could post of a simple logic problem which I asked ChatGPT to solve step by step. It started off okay, but then it started completely breaking the rules of the puzzle and the internal consistency of its answer. When I asked it to identify its mistake, it made up a totally different mistake it never made. The thing is, it's not emulating. It's imitating, and there's a big difference: When you emulate a logical process, you're copying it from the bottom up, and the internal logic is the same. If there are mistakes, they're fairly marginal. When you imitate a process, however, you're trying to reproduce the same output without recreating the internal logic that creates that output, and it's easy to detect and extrapolate patterns in the output in ways that don't make sense (like with the 12L/6L jug puzzle).
  - pixl97 3y ago
    > it can still contradict itself Do humans not quite often contradict themselves?
    - bccdee 3y ago
      Not the way LLMs do. Typically, humans are only hypocritical when they're trying to conceal their true values. For example, when someone makes a "states' rights for me but not for thee" argument, they don't actually believe in "states' rights." They're just cynically arguing whatever position will forward their agenda. An LLM, conversely, will sometimes directly gainsay itself, hallucinating different versions of its own output from just paragraphs earlier. That's not just hypocrisy—that's not having any actual concept of truth beyond an imperfect idea of when a sentence does or doesn't look "realistic."
      - famouswaffles 3y ago
        >Typically, humans are only hypocritical when they're trying to conceal their true values. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/ People sure as hell think they know what previous mental states and preferences inform their decisions but that's just as likely to be nice post-hoc rationalization than truth. You're a co-pilot at best. You don't actually know why you make the decisions you do. No need for any "concealing my true self" rhetoric
        bccdee 3y ago
        The article you linked has nothing to do with your claim and neither the article nor your claim have anything to do with my point. The article demonstrates proof of choice-supportive bias, not that people have no recollection of their previous mental states. And LLMs don't merely exhibit choice-supportive bias—they wantonly contradict things they said moments earlier and wildly hallucinate things that never happened.
        3y ago
- yosito 3y ago
  Hiding part of the LLMs output from the user doesn't make what it's doing fundamentally different than what it does now. Also, anecdotally, I can totally reason about things non-verbally in my mind and then verbally express a conclusion. Not everything is an internal dialogue for me. But so far, everything has to be an internal (or external) dialogue for an LLM or it can't reason at all.
  - bccdee 3y ago
    Some people don't have an inner monologue at all, and reason entirely non-verbally except when they're writing or speaking aloud. https://www.cbc.ca/news/canada/saskatchewan/inner-monologue-experience-science-1.5486969 https://www.cbc.ca/news/canada/saskatchewan/inner-monologue-...

by bccdee 3y ago

oezi 3y ago

I see absolutely no reason (except cost) that LLM interfaces shouldn't utilize an internal dialog which is hidden from the user and which can be queried from the first layer of interaction and potentially vice versa. This would emulate much of what we consider human reasoning. And if it quacks like a duck it might be a duck...

bccdee 3y ago
Asking the LLM to be more verbose can bring contradictions to the surface and lead to more reliable answers, but it still doesn't fix the underlying problem. Even when you ask it to show its work, it can still contradict itself, blindly imitate patterns in its training data, and contrive plausible-sounding but false answers. I've got a (rather long) transcript I could post of a simple logic problem which I asked ChatGPT to solve step by step. It started off okay, but then it started completely breaking the rules of the puzzle and the internal consistency of its answer. When I asked it to identify its mistake, it made up a totally different mistake it never made. The thing is, it's not emulating. It's imitating, and there's a big difference: When you emulate a logical process, you're copying it from the bottom up, and the internal logic is the same. If there are mistakes, they're fairly marginal. When you imitate a process, however, you're trying to reproduce the same output without recreating the internal logic that creates that output, and it's easy to detect and extrapolate patterns in the output in ways that don't make sense (like with the 12L/6L jug puzzle).
- pixl97 3y ago
  > it can still contradict itself Do humans not quite often contradict themselves?
  - bccdee 3y ago
    Not the way LLMs do. Typically, humans are only hypocritical when they're trying to conceal their true values. For example, when someone makes a "states' rights for me but not for thee" argument, they don't actually believe in "states' rights." They're just cynically arguing whatever position will forward their agenda. An LLM, conversely, will sometimes directly gainsay itself, hallucinating different versions of its own output from just paragraphs earlier. That's not just hypocrisy—that's not having any actual concept of truth beyond an imperfect idea of when a sentence does or doesn't look "realistic."
    - famouswaffles 3y ago
      >Typically, humans are only hypocritical when they're trying to conceal their true values. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/ People sure as hell think they know what previous mental states and preferences inform their decisions but that's just as likely to be nice post-hoc rationalization than truth. You're a co-pilot at best. You don't actually know why you make the decisions you do. No need for any "concealing my true self" rhetoric
      - bccdee 3y ago
        The article you linked has nothing to do with your claim and neither the article nor your claim have anything to do with my point. The article demonstrates proof of choice-supportive bias, not that people have no recollection of their previous mental states. And LLMs don't merely exhibit choice-supportive bias—they wantonly contradict things they said moments earlier and wildly hallucinate things that never happened.
        3y ago
yosito 3y ago
Hiding part of the LLMs output from the user doesn't make what it's doing fundamentally different than what it does now. Also, anecdotally, I can totally reason about things non-verbally in my mind and then verbally express a conclusion. Not everything is an internal dialogue for me. But so far, everything has to be an internal (or external) dialogue for an LLM or it can't reason at all.
- bccdee 3y ago
  Some people don't have an inner monologue at all, and reason entirely non-verbally except when they're writing or speaking aloud. https://www.cbc.ca/news/canada/saskatchewan/inner-monologue-experience-science-1.5486969 https://www.cbc.ca/news/canada/saskatchewan/inner-monologue-...