>For reasoning of any complexity, some sort of internal monologue and iteration is needed. From a formal perspective you're entirely correct. Transformers with

>For reasoning of any complexity, some sort of internal monologue and iteration is needed. From a formal perspective you're entirely correct. Transformers with chain-of-thought are strictly more powerful than transformers without it, and can efficiently solve classes of problems that would otherwise require exponentially increasing model depth: https://arxiv.org/abs/2310.07923 https://arxiv.org/abs/2310.07923