6 ms·
If you ask humans to explain why we did something, Sperry's split brain experiment gives reason to think you can't trust our accounts of why we did something ei
by vidarh 6mo ago
If you ask humans to explain why we did something, Sperry's split brain experiment gives reason to think you can't trust our accounts of why we did something either (his experiments showed the brain making up justifications for decisions it never made)
Bit it can still be useful, as long as you interpret it as "which stimuli most likely triggered the behaviour?" You can't trust it uncritically, but models do sometimes pinpoint useful things about how they were prompted.
- emp17344 6mo agoThat is absolutely not what the split brain experiment reveals. Why would you take results received from observing the behavior of a highly damaged brain, and use them to predict the behavior of a healthy brain? Stop spreading misinformation.
- nuancebydefault 6mo agoSuch 'highly damaged' brain is still 90 percent or more structured the same as a normal human brain. See it as a brain that runs in debug mode. It is known that the narrative part of the brain is separate from the decision taking brain. If someone asks you, in a very convincing, persuasive way, why you did something a year ago and you can't clearly remember you did, it can happen that you become positive that you did so anyway. And then the mind just hallucinates a reason. That's a trait of brains.
- Jensson 6mo ago> If someone asks you, in a very convincing, persuasive way, why you did something a year ago and you can't clearly remember you did, it can happen that you become positive that you did so anyway. And then the mind just hallucinates a reason. That's a trait of brains. Yes brains can hallucinate reasons, doesn't mean they always do. If all reasons given were hallucinations then introspection would be impossible, but clearly introspection do help people.
- vidarh 6mo agoBecause said "highly damaged brain" in most respects still functions pretty much like a healthy one. There is no misinformation in what I wrote.
- pierrekin 6mo agoI agree that the model can help troubleshoot and debug itself. I argue that the model has no access to its thoughts at the time. Split brain experiments notwithstanding I believe that I can remember what my faulty assumptions were when I did something. If you ask a model “why did you do that” it is literally not the same “brain instance” anymore and it can only create reasons retroactively based on whatever context it recorded (chain of thought for example).
- jmalicki 6mo agoIt does have access to its thoughts. This is literally what thinking models do. They write out thoughts to a scratch pad (which you can see!) and use that as part of the prompt.
- deleted 6mo ago[deleted]
- grey-area 6mo agoThey do not in fact do that. The ‘thoughts’ are not a chain of logic.
- mmoll 6mo agoIt doesn’t mean that these “thoughts” influenced their final decision the way they would in humans. An LLM will tell you a lot of things it “considered” and its final output might still be completely independent of that.
- jmalicki 6mo agoIts output quite literally is not independent, as the "thinking tokens" are attended to by the attention mechanism.
- fc417fc802 6mo agoIt's important to be aware that while those "thoughts" can be a useful aid for human understanding they don't seem to reliably reflect what's going on under the hood. There are various academic papers on the matter or you can closely inspect the traces of a more logically oriented question for yourself and spot impossible inconsistencies.
- cmiles74 6mo agoNone of the developers that I’ve worked with have had the hemispheres of their brains severed. I suspect this is pretty rare in the field.
- pixl97 6mo agoThis still doesnt stop post ad hoc explanations by humans.
- deleted 6mo ago[deleted]
- tempaccount5050 6mo agoI feel like your conflating a deep misconfiguration of a brain with lying. These things are completely different.
- lmm 6mo ago> None of the developers that I’ve worked with have had the hemispheres of their brains severed. But are their explanations for how they behaved any more compelling than those of people who have? If so, why?
- amluto 6mo agoHumans can do one thing that AI agents are 100% completely incapable of doing: being accountable for their actions.
- jeremyccrane 6mo agoYep.
- grey-area 6mo agoDon’t forget learning, humans can learn, LLMs do not learn, they are trained before use.
- addedGone 6mo agoThey learn on the next update :p
- quantummagic 6mo agoYup. And eventually there will be online learning, that doesn't require a formal update step. People keep conflating the current implementation, as an inherent feature.
- grey-area 6mo agoThat’s training, not learning.
- HighGoldstein 6mo agoDo we? Or are we born with pre-training (all the crucial functions the brain does without us having to learn them) and a context window orders of magnitude larger than an LLM?
- compass_copium 6mo agoIt is incredible how willing and eager AI boosters are to denigrate the incredible miracle of human consciousness to make their chatbots seem so special. No, we are not born with all the pre-training we need. That is rather the point of education, teaching people's brains how to process information in new, maybe unintuitive ways.
- jayd16 6mo agoYou might as well be asking a tape recorder why it said something. Why are we confusing the situation with non-nonsensical comparisons? There is no internal monologue with which to have introspection (beyond what the AI companies choose to hide as a matter of UX or what have you). There is no "I was feeling upset when I said/did that" unless it's in the context. There is no ghost in the machine that we cannot see before asking. Even if a model is able to come up with a narrative, it's simply that. Looking at the log and telling you a story.
- vidarh 6mo agoSperry's experiments makes it quite clear that the comparison is not nonsensical: humans can't reliably tell why we do things either. It is not imbuing AI with anything more to recognise that. Rather pointing out that when we seek to imply the gap is so huge we often overestimate our own abilities.
- abcde666777 6mo agoSlight pushback - I think there's still a lot more consistency and coherence in a human's recollection of their motives than an LLM. Sometimes I think we're too eager to compare ourselves to them.
- vidarh 6mo agoWe have pretty much evidence to support that human recollection includes the right data to be able to ascertain why we actually did something.
- fluoridation 6mo agoHumans at least have a mental state that only they are privy to to work from, and not just their words and actions. The LLM literally cannot possibly have a deeper insight into the root cause than the user, because it can only work from the information that the user has access to.
- 6mo ago
- tempaccount5050 6mo agoI think you might be misinterpreting that. I always understood it to mean that when the two hemispheres can't communicate, they'll make things up about their unknowable motivations to basically keep consciousness in a sane state (avoiding a kernel panic?). I don't think it's clear that this happens when both hemispheres are able to communicate properly. At least, I don't think you can imply that this special case is applicable all the time.
- vidarh 6mo agoWe have no reason to believe it is a special case. The fact that these patients largely functioned normally when you did not create a situation preventing the hemispheres from synchronising suggests otherwise to me. There's no reason to think the ability to just make things up and treat it as if it is truthful recollection would just disappear because there are two halves that can lie instead of just one.
- layer8 6mo agoThe thing is, the LLM mostly just states what it did, and doesn't really explain it (other than "I didn't understand what I was doing before doing it. I didn't read Railway's docs on volume behavior across environments."). Humans are able of more introspection, and usually have more awareness of what leads them to do (or fail to do) things. LLMs are lacking layers of awareness that humans have. I wonder if achieving comparable awareness in LLMs would require significantly more compute, and/or would significantly slow them down.
- vidarh 6mo agoSperry's experiments suggests we don't have that awareness, but think we do as our brains will make up an explanation on the spot.