AFAIU, that's basically impossible with a typical transformer architecture. The "recursive" transforms mentioned above are indeed a little closer, but still. Yo

AFAIU, that's basically impossible with a typical transformer architecture. The "recursive" transforms mentioned above are indeed a little closer, but still. You could just always start a new inference after the last one finishes, of course, but that feels more like a really fast, one-sided conversation than a real approximation of the modality/lifecycle of human consciousness. I think the main answer to this concern is that humans are absolutely scheduling wake-ups on a cognitive level -- the best example may be, y'know, sleep! But also on a moment-to-moment basis, which is especially noticable during periods of boredom. Think of the head LLM as you, and the workers as your subconscious faculties (e.g. the part of you that knows how to ride a bike in ways that you have never had to consciously articulate). The looping part is the unconscious substrate that makes all of that possible, arguably with some room for the faculty above you (metacognition) to control what gets presented to your conscious mind and when. The vast, vast majority of input never makes it that far tho, by design. (ETA: ...so, that means that we don't need to fundamentally change the architecture of LLMs in order to get some really scary stuff going.)