It's just 'prediction error' in a feedback loop, imo. I'm sure like any other biological human with mitochondria and stuff, you've occasionally said (or starte

It's just 'prediction error' in a feedback loop, imo. I'm sure like any other biological human with mitochondria and stuff, you've occasionally said (or started to say) something and then you (ie. the actively cross-checking self-analyzing enigma that is 'you') thinks 'hang on, no that doesn't make sense' and you self-correct. LLMs are 100% feedforward, there's just one big autoregression going on. No strange loop shenanigans. Honestly I'm really interested to see where LLM-based diffusion models end up. (To be fair, probably mostly because I don't understand them yet so they could still be spooky. :D )