4 ms·

>That's not what's going on. GPT4 can't know the real reason why it wrote anything. Boy do I have news for you. People can't recreate previous mental states,

>That's not what's going on. GPT4 can't know the real reason why it wrote anything. Boy do I have news for you. People can't recreate previous mental states, it's all plausible post-hoc rationalization. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/ And the rationalizations aren't always grounded either. The brain is just fine making up completely bogus explanations you believe to be true but couldn't possibly be so. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4204522/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4204522/ https://pure.uva.nl/ws/files/25987577/Split_Brain.pdf https://pure.uva.nl/ws/files/25987577/Split_Brain.pdf The conscious self is at best a co-pilot. A Co-pilot that has shockingly little insight into the operations of the plane but thinks he knows much more. Even your senses gets some pretty big post-processing by your brain you're consciously unaware of. For example, ever wondered why the first second seems longer when you suddenly look at a clock ? Human beings may not have the best visual acuity, but we are quite good at distinguishing colors and exceptional at processing visual information. A significant portion of our brain is dedicated to processing the visual stimuli from our binocular vision. Nevertheless, binocular vision necessitates that both eyes focus on a single point. Our eyes can track a focused target as it moves, but they are unable to smoothly scan across our visual field. This is too difficult to process, so our eyes naturally jump from one point to another. Yet, during these jumps, known as "saccades," our brain does not even attempt to comprehend the blurry sensations that come in. In effect, you temporarily go completely blind! However, you don't feel like you went blind, right? Your vision doesn't go black or anything, and it seems like you could see the entire time. This is because our brains fill in the lag while our eyes are moving, creating the illusion that we were seeing something instead of being blind during that time. Strangely enough, our brains fill it in with *what we see when they stop*! In other words, our brains go back and edit our memory of a split second ago to make it seem like we were looking at our new point of focus the whole time our eyes were moving to actually see it. This phenomenon occurs when you look at the second hand of a clock. When you glance over at the clock, your brain tells you that you were looking at it for longer than you actually were, adding the time your eyes were moving but not yet seeing the clock. If the second hand was moving during this time, your brain will believe the second hand was in its new position longer than it actually was. Based on this flawed memory, it will appear as if the second hand stayed in that position for longer than a second.

skybrian 4y ago
Yes, of course. We often don't know our real motivations either, our justifications are often self-serving and made up, and more generally there's a lot we don't know about how the brain works. But we do at least have some memories of our own thought processes and some experience observing our own reactions to things. You can get better at introspection. Attempting to understand and explain your own mistakes is useful. An LLM won't be able to do this sort of learning without changing its training or architecture somehow.
- famouswaffles 4y ago
  Actually just added a long paragraph showing how even memories or real time sense data is pleasant fabrication. Also every time you remember something, you're not really recalling it. Your brain rewrites your memory every time you "remember" often introducing false data or inconsistencies. It's one of the reasons human memory is so fallible and inadmissible. That said, I don't disagree with the overall point of what you've said. Attempted Introspection is useful. The point I'm making is to not dismiss LLM "introspection" simply because it's rationalization. It can be extremely useful too.
  - skybrian 4y ago
    Fair enough! The part I see missing for an LLM is learning from feedback about whether its justifications "work" for explaining its own actions. Keep in mind that an LLM is not human and its justifications often shouldn't look like human justifications. But it's going to be scored by how human its justifications look, so you will never get that. (With the exception of things explicitly trained in like knowing its own cut-off date - that's quite a non-human justification for not knowing something.) Also, it's easy to come up with examples showing that we do remember what we were thinking, even if we didn't say it out loud. A simple example might be running an errand. Of course, sometimes we do forget, which is why a shopping list is useful. An LLM cannot do that at all. It writes it down or it's lost. It can only pretend to remember an internal thought it had.
    - 4y ago

skybrian 4y ago

Yes, of course. We often don't know our real motivations either, our justifications are often self-serving and made up, and more generally there's a lot we don't know about how the brain works. But we do at least have some memories of our own thought processes and some experience observing our own reactions to things. You can get better at introspection. Attempting to understand and explain your own mistakes is useful. An LLM won't be able to do this sort of learning without changing its training or architecture somehow.

famouswaffles 4y ago
Actually just added a long paragraph showing how even memories or real time sense data is pleasant fabrication. Also every time you remember something, you're not really recalling it. Your brain rewrites your memory every time you "remember" often introducing false data or inconsistencies. It's one of the reasons human memory is so fallible and inadmissible. That said, I don't disagree with the overall point of what you've said. Attempted Introspection is useful. The point I'm making is to not dismiss LLM "introspection" simply because it's rationalization. It can be extremely useful too.
- skybrian 4y ago
  Fair enough! The part I see missing for an LLM is learning from feedback about whether its justifications "work" for explaining its own actions. Keep in mind that an LLM is not human and its justifications often shouldn't look like human justifications. But it's going to be scored by how human its justifications look, so you will never get that. (With the exception of things explicitly trained in like knowing its own cut-off date - that's quite a non-human justification for not knowing something.) Also, it's easy to come up with examples showing that we do remember what we were thinking, even if we didn't say it out loud. A simple example might be running an errand. Of course, sometimes we do forget, which is why a shopping list is useful. An LLM cannot do that at all. It writes it down or it's lost. It can only pretend to remember an internal thought it had.
  - 4y ago