3 ms·
> Brains do this in spades. Sense data you perceive is partly fabricated for the concious self. You must be sprinting to move the goalposts that far that fast.
by bccdee 3y ago
> Brains do this in spades. Sense data you perceive is partly fabricated for the concious self.
You must be sprinting to move the goalposts that far that fast. I'm not talking about convenient sensory illusions or details of perception. I'm talking about ChatGPT telling bold-faced and obvious lies about things it said mere sentences earlier. Would you engage with my points instead of switching to totally unrelated topics like Hitler's single Jewish friend?
Humans are not perfectly rational beings and our memory is faulty, but we are broadly rational and have a generally consistent model of the world; LLMs are painfully inconsistent and exhibits wild bouts of irrationality, and this behaviour is totally in line with the way they are trained.
- oezi 3y agoI think it is a mistake to consider this an inherent weakness of LLMs. Nothing precludes a system architecture for a next generation LLM to have loops for rationality or morality reasoning which filter the output. We are likely to see many human psychotic disorders arising in the LLM output when we start doing this but we will likely see less of the pivoting on answers we see now.
- bccdee 3y agoNo, you're massively anthropomorphizing them. You can't just add a "loop for rationality"—writing is a second-order product of first-order reasoning. If a model trained on that second-order product isn't reproducing the first-order behaviour, you can't just tack it on in an extra module. It's like trying to teach something to dance based on the footprints of dancers. Depending on the dance, maybe it'll work and maybe not, but with the dance of reason and the footprints of writing it's not working, and no extra module we tack on will change the relationship between the dance and the footprints. It's a fundamental flaw in the LLM architecture.