5 ms·

I see absolutely no reason (except cost) that LLM interfaces shouldn't utilize an internal dialog which is hidden from the user and which can be queried from th

by oezi 3y ago

I see absolutely no reason (except cost) that LLM interfaces shouldn't utilize an internal dialog which is hidden from the user and which can be queried from the first layer of interaction and potentially vice versa. This would emulate much of what we consider human reasoning. And if it quacks like a duck it might be a duck...

bccdee 3y ago
Asking the LLM to be more verbose can bring contradictions to the surface and lead to more reliable answers, but it still doesn't fix the underlying problem. Even when you ask it to show its work, it can still contradict itself, blindly imitate patterns in its training data, and contrive plausible-sounding but false answers. I've got a (rather long) transcript I could post of a simple logic problem which I asked ChatGPT to solve step by step. It started off okay, but then it started completely breaking the rules of the puzzle and the internal consistency of its answer. When I asked it to identify its mistake, it made up a totally different mistake it never made. The thing is, it's not emulating. It's imitating, and there's a big difference: When you emulate a logical process, you're copying it from the bottom up, and the internal logic is the same. If there are mistakes, they're fairly marginal. When you imitate a process, however, you're trying to reproduce the same output without recreating the internal logic that creates that output, and it's easy to detect and extrapolate patterns in the output in ways that don't make sense (like with the 12L/6L jug puzzle).
- pixl97 3y ago
  > it can still contradict itself Do humans not quite often contradict themselves?
  - bccdee 3y ago
    Not the way LLMs do. Typically, humans are only hypocritical when they're trying to conceal their true values. For example, when someone makes a "states' rights for me but not for thee" argument, they don't actually believe in "states' rights." They're just cynically arguing whatever position will forward their agenda. An LLM, conversely, will sometimes directly gainsay itself, hallucinating different versions of its own output from just paragraphs earlier. That's not just hypocrisy—that's not having any actual concept of truth beyond an imperfect idea of when a sentence does or doesn't look "realistic."
    - famouswaffles 3y ago
      >Typically, humans are only hypocritical when they're trying to conceal their true values. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/ People sure as hell think they know what previous mental states and preferences inform their decisions but that's just as likely to be nice post-hoc rationalization than truth. You're a co-pilot at best. You don't actually know why you make the decisions you do. No need for any "concealing my true self" rhetoric
      - bccdee 3y ago
        The article you linked has nothing to do with your claim and neither the article nor your claim have anything to do with my point. The article demonstrates proof of choice-supportive bias, not that people have no recollection of their previous mental states. And LLMs don't merely exhibit choice-supportive bias—they wantonly contradict things they said moments earlier and wildly hallucinate things that never happened.
        famouswaffles 3y ago
        I didn't say people have no recollection of previous mental states. I said people don't know what mental states and preferences actually inform decisions. They just think they do. And previous mental states can't be recreated. Hitler has his share of Jews saved from the Holocaust. For example, he befriended a Jew girl, Rosa Bernile Nienau among a few others and kept regular contact with her. So tell me, what "secret self" was Hitler hiding ? Humanity is chock full of contradictory actions. This nonsense "secret self" theory doesn't even make sense for most of them. >they wantonly contradict things they said moments earlier and wildly hallucinate things that never happened. Brains do this in spades. Sense data you perceive is partly fabricated for the concious self. This is why you see a second tick by faster when you suddenly look at the clock. This is why it seems you see something when you shift your eyes quickly instead of the completely black you should be seeing. Memories are always part-fabrications too. Only what the brain perceives as "key" is stored, making up the rest. Every time you recall a memory, the brain rewrites part of it, often introducing more fabrications. It's extremely easy to implant targeted false memories for this exact reason. https://en.m.wikipedia.org/wiki/Lost_in_the_mall_technique https://en.m.wikipedia.org/wiki/Lost_in_the_mall_technique And then there's the results of the split brain experiments too https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7305066/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7305066/ So this idea that your brain doesn't "hallucinate" or make up details is very laughable. Your brain is quite literally constantly doing that and with the concious mind completely unaware.
yosito 3y ago
Hiding part of the LLMs output from the user doesn't make what it's doing fundamentally different than what it does now. Also, anecdotally, I can totally reason about things non-verbally in my mind and then verbally express a conclusion. Not everything is an internal dialogue for me. But so far, everything has to be an internal (or external) dialogue for an LLM or it can't reason at all.
- bccdee 3y ago
  Some people don't have an inner monologue at all, and reason entirely non-verbally except when they're writing or speaking aloud. https://www.cbc.ca/news/canada/saskatchewan/inner-monologue-experience-science-1.5486969 https://www.cbc.ca/news/canada/saskatchewan/inner-monologue-...

by oezi 3y ago

bccdee 3y ago

Asking the LLM to be more verbose can bring contradictions to the surface and lead to more reliable answers, but it still doesn't fix the underlying problem. Even when you ask it to show its work, it can still contradict itself, blindly imitate patterns in its training data, and contrive plausible-sounding but false answers. I've got a (rather long) transcript I could post of a simple logic problem which I asked ChatGPT to solve step by step. It started off okay, but then it started completely breaking the rules of the puzzle and the internal consistency of its answer. When I asked it to identify its mistake, it made up a totally different mistake it never made. The thing is, it's not emulating. It's imitating, and there's a big difference: When you emulate a logical process, you're copying it from the bottom up, and the internal logic is the same. If there are mistakes, they're fairly marginal. When you imitate a process, however, you're trying to reproduce the same output without recreating the internal logic that creates that output, and it's easy to detect and extrapolate patterns in the output in ways that don't make sense (like with the 12L/6L jug puzzle).

pixl97 3y ago
> it can still contradict itself Do humans not quite often contradict themselves?
- bccdee 3y ago
  Not the way LLMs do. Typically, humans are only hypocritical when they're trying to conceal their true values. For example, when someone makes a "states' rights for me but not for thee" argument, they don't actually believe in "states' rights." They're just cynically arguing whatever position will forward their agenda. An LLM, conversely, will sometimes directly gainsay itself, hallucinating different versions of its own output from just paragraphs earlier. That's not just hypocrisy—that's not having any actual concept of truth beyond an imperfect idea of when a sentence does or doesn't look "realistic."
  - famouswaffles 3y ago
    >Typically, humans are only hypocritical when they're trying to conceal their true values. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/ People sure as hell think they know what previous mental states and preferences inform their decisions but that's just as likely to be nice post-hoc rationalization than truth. You're a co-pilot at best. You don't actually know why you make the decisions you do. No need for any "concealing my true self" rhetoric
    - bccdee 3y ago
      The article you linked has nothing to do with your claim and neither the article nor your claim have anything to do with my point. The article demonstrates proof of choice-supportive bias, not that people have no recollection of their previous mental states. And LLMs don't merely exhibit choice-supportive bias—they wantonly contradict things they said moments earlier and wildly hallucinate things that never happened.
      - famouswaffles 3y ago
        I didn't say people have no recollection of previous mental states. I said people don't know what mental states and preferences actually inform decisions. They just think they do. And previous mental states can't be recreated. Hitler has his share of Jews saved from the Holocaust. For example, he befriended a Jew girl, Rosa Bernile Nienau among a few others and kept regular contact with her. So tell me, what "secret self" was Hitler hiding ? Humanity is chock full of contradictory actions. This nonsense "secret self" theory doesn't even make sense for most of them. >they wantonly contradict things they said moments earlier and wildly hallucinate things that never happened. Brains do this in spades. Sense data you perceive is partly fabricated for the concious self. This is why you see a second tick by faster when you suddenly look at the clock. This is why it seems you see something when you shift your eyes quickly instead of the completely black you should be seeing. Memories are always part-fabrications too. Only what the brain perceives as "key" is stored, making up the rest. Every time you recall a memory, the brain rewrites part of it, often introducing more fabrications. It's extremely easy to implant targeted false memories for this exact reason. https://en.m.wikipedia.org/wiki/Lost_in_the_mall_technique https://en.m.wikipedia.org/wiki/Lost_in_the_mall_technique And then there's the results of the split brain experiments too https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7305066/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7305066/ So this idea that your brain doesn't "hallucinate" or make up details is very laughable. Your brain is quite literally constantly doing that and with the concious mind completely unaware.

yosito 3y ago

Hiding part of the LLMs output from the user doesn't make what it's doing fundamentally different than what it does now. Also, anecdotally, I can totally reason about things non-verbally in my mind and then verbally express a conclusion. Not everything is an internal dialogue for me. But so far, everything has to be an internal (or external) dialogue for an LLM or it can't reason at all.

bccdee 3y ago
Some people don't have an inner monologue at all, and reason entirely non-verbally except when they're writing or speaking aloud. https://www.cbc.ca/news/canada/saskatchewan/inner-monologue-experience-science-1.5486969 https://www.cbc.ca/news/canada/saskatchewan/inner-monologue-...