4 ms·

> People can't recreate previous mental states, it's all plausible post-hoc rationalization. [...] How could you possibly claim people are simply incapable of

> People can't recreate previous mental states, it's all plausible post-hoc rationalization. [...] How could you possibly claim people are simply incapable of recreating mental states and that all they do is plausible post-hoc rationalization? Surely people can sometimes remember what they were thinking before making a decision? And in fact I would think people can do this more frequently than GPT can. Or are you really claiming those studies proved an implausible negative here?

throwawayadvsec 3y ago
"what they were thinking before making a decision?" Human brains are notoriously unreliable, we lie to ourselves all the time, we can't trust our memories.
- dataflow 3y ago
  > "what they were thinking before making a decision?" Human brains are notoriously unreliable, we lie to ourselves all the time, we can't trust our memories. "Unreliable" is a far cry from "lie to ourselves all the time". Some of the time, sure. But all the time? Everything I'm telling myself is a post-hoc fabrication just like with GPT? Like you're telling me I can't even trust my memory of who my mom and dad are, or which country I live in? I'm justifying all of these post-facto somehow?
  - 20after4 3y ago
    Perhaps those specific details are just a few heavily reinforced data points. Perhaps they become more and more statistically likely to be true the more they are reinforced?
    - dataflow 3y ago
      Or perhaps they aren't, who knows. Regardless, is this a hypothesis that leads to anything useful?
famouswaffles 3y ago
The sense data you consciously perceive is pleasant fabrication. Moreover, every time you remember something, you're not really recalling it. Your brain rewrites your memory every time you "remember" (what you're really doing is remembering the last time you remembered it), often introducing false data or inconsistencies. It's one of the reasons human memory is so fallible and inadmissible. The reason I linked the split brain experiments is to show that what you think is the reason for making a decision, even if you "remember" the thought processes before can just be wrong. Asking one side to perform an action and asking the other side to explain the reason for performing the action ended up producing some really wild results.
- dataflow 3y ago
  > The reason I linked the split brain experiments is to show that what you think is the reason for making a decision, even if you "remember" the thought processes before can just be wrong. Asking one side to perform an action and asking the other side to explain the reason for performing the action ended up producing some really wild results. "Can" just be wrong and "produces some" wild results, sure. In the previous comment were claiming they're always fabrications (as, I note, seems to be the case with GPT!), not just that they could be. The fact that both GPT and the human brain can sometimes fabricate things similarly does not mean the human brain (or GPT for that matter) does this all the time, let alone that they do this equally frequently, or always to equal degrees. The studies might be showing surprising results, but you're greatly exaggerating what the studies actually show.
  - famouswaffles 3y ago
    Memories are always part-fabrications. Our brain store "key" moments it thinks it is important, then make up the rest of it. But this process is not really trustworthy on the finer details and is extremely receptive to any sort of suggestion because it simply doesn't have any factual data, so happily jumps to anything which seems to fit the narrative. This is especially true in scenarios where you experience strong emotions. Anything feels like to fit (like, someone paints a scenario which creates the same type of emotion) our brain just accepts it as a matter of fact and puts the puzzle in. It's why implanting false memories is extremely easy and leading questions work so well. https://en.m.wikipedia.org/wiki/Lost_in_the_mall_technique https://en.m.wikipedia.org/wiki/Lost_in_the_mall_technique Anything that you remember from your life is mostly fabrications with a couple of actual memories used as a scaffolding, and your brain just built the rest of the structure around it - constantly changing it.
sebzim4500 3y ago
Exactly, the people making confident statements about the inner workings of the brain are just as ridiculous as the people making the same statements about LLMs. We should have a healthy dose of humility here and realize we know very little about either the brain or the meaning hidden in those ~200 billion parameters.
chaorace 3y ago
First: these studies are unrelated to AI and mostly predate modern LLMs, so no, they draw no direct comparisons. At best, this is pointing out two surprisingly similar behaviors we observe in LLMs and the actual functioning of the human mind. Second: Being unable to reconstruct a prior mental state =/= remembering a thought process. I'll use an accursed programing analogy because we are on an accursed programing website: it's like the difference between debug logs and time-travel debugging. In plain language: humans retroactively reason about their thought process based on the artifacts produced by that thought process. The quality and reliability of these artifacts will vary heavily, but the reasoning itself is always backward-looking and speculative in nature.
- dataflow 3y ago
  That's all fine, but if the claim is really "I cannot know which exact neurons were firing at some moment in the past" (that's what time-travel debugging would be), that's not a particularly insightful or useful statement. I'm pretty sure I can reconstruct some states of my brain for some moments in the past sufficiently adequately to be able to explain my reasoning in that moment without fabricating anything. The claim was that this is always impossible for all humans, which seems quite obviously false, and in any case, unsupported by the studies linked.
  - pixl97 3y ago
    Honestly the proof of evidence that you can recall your memory state needs to be proven by you, as I don't believe any one has brought forth evidence of what you're suggesting. Almost everything I've seen and read about this suggests the opposite. Activating memory changes memory. Intermediate events between the initial activation and the final questioning can also change those memories. And furthermore, that you are only aware of your conscious memories (and even most of that is forgotten) and that memory is accessed unconsciously all the time. It's all post ad hoc.

throwawayadvsec 3y ago

"what they were thinking before making a decision?" Human brains are notoriously unreliable, we lie to ourselves all the time, we can't trust our memories.

dataflow 3y ago
> "what they were thinking before making a decision?" Human brains are notoriously unreliable, we lie to ourselves all the time, we can't trust our memories. "Unreliable" is a far cry from "lie to ourselves all the time". Some of the time, sure. But all the time? Everything I'm telling myself is a post-hoc fabrication just like with GPT? Like you're telling me I can't even trust my memory of who my mom and dad are, or which country I live in? I'm justifying all of these post-facto somehow?
- 20after4 3y ago
  Perhaps those specific details are just a few heavily reinforced data points. Perhaps they become more and more statistically likely to be true the more they are reinforced?
  - dataflow 3y ago
    Or perhaps they aren't, who knows. Regardless, is this a hypothesis that leads to anything useful?

famouswaffles 3y ago

The sense data you consciously perceive is pleasant fabrication. Moreover, every time you remember something, you're not really recalling it. Your brain rewrites your memory every time you "remember" (what you're really doing is remembering the last time you remembered it), often introducing false data or inconsistencies. It's one of the reasons human memory is so fallible and inadmissible. The reason I linked the split brain experiments is to show that what you think is the reason for making a decision, even if you "remember" the thought processes before can just be wrong. Asking one side to perform an action and asking the other side to explain the reason for performing the action ended up producing some really wild results.

dataflow 3y ago
> The reason I linked the split brain experiments is to show that what you think is the reason for making a decision, even if you "remember" the thought processes before can just be wrong. Asking one side to perform an action and asking the other side to explain the reason for performing the action ended up producing some really wild results. "Can" just be wrong and "produces some" wild results, sure. In the previous comment were claiming they're always fabrications (as, I note, seems to be the case with GPT!), not just that they could be. The fact that both GPT and the human brain can sometimes fabricate things similarly does not mean the human brain (or GPT for that matter) does this all the time, let alone that they do this equally frequently, or always to equal degrees. The studies might be showing surprising results, but you're greatly exaggerating what the studies actually show.
- famouswaffles 3y ago
  Memories are always part-fabrications. Our brain store "key" moments it thinks it is important, then make up the rest of it. But this process is not really trustworthy on the finer details and is extremely receptive to any sort of suggestion because it simply doesn't have any factual data, so happily jumps to anything which seems to fit the narrative. This is especially true in scenarios where you experience strong emotions. Anything feels like to fit (like, someone paints a scenario which creates the same type of emotion) our brain just accepts it as a matter of fact and puts the puzzle in. It's why implanting false memories is extremely easy and leading questions work so well. https://en.m.wikipedia.org/wiki/Lost_in_the_mall_technique https://en.m.wikipedia.org/wiki/Lost_in_the_mall_technique Anything that you remember from your life is mostly fabrications with a couple of actual memories used as a scaffolding, and your brain just built the rest of the structure around it - constantly changing it.

sebzim4500 3y ago

Exactly, the people making confident statements about the inner workings of the brain are just as ridiculous as the people making the same statements about LLMs. We should have a healthy dose of humility here and realize we know very little about either the brain or the meaning hidden in those ~200 billion parameters.

chaorace 3y ago

First: these studies are unrelated to AI and mostly predate modern LLMs, so no, they draw no direct comparisons. At best, this is pointing out two surprisingly similar behaviors we observe in LLMs and the actual functioning of the human mind. Second: Being unable to reconstruct a prior mental state =/= remembering a thought process. I'll use an accursed programing analogy because we are on an accursed programing website: it's like the difference between debug logs and time-travel debugging. In plain language: humans retroactively reason about their thought process based on the artifacts produced by that thought process. The quality and reliability of these artifacts will vary heavily, but the reasoning itself is always backward-looking and speculative in nature.

dataflow 3y ago
That's all fine, but if the claim is really "I cannot know which exact neurons were firing at some moment in the past" (that's what time-travel debugging would be), that's not a particularly insightful or useful statement. I'm pretty sure I can reconstruct some states of my brain for some moments in the past sufficiently adequately to be able to explain my reasoning in that moment without fabricating anything. The claim was that this is always impossible for all humans, which seems quite obviously false, and in any case, unsupported by the studies linked.
- pixl97 3y ago
  Honestly the proof of evidence that you can recall your memory state needs to be proven by you, as I don't believe any one has brought forth evidence of what you're suggesting. Almost everything I've seen and read about this suggests the opposite. Activating memory changes memory. Intermediate events between the initial activation and the final questioning can also change those memories. And furthermore, that you are only aware of your conscious memories (and even most of that is forgotten) and that memory is accessed unconsciously all the time. It's all post ad hoc.