Transformers with memory can abstract recursively in theory. https://arxiv.org/abs/2301.04589 https://arxiv.org/abs/2301.04589. If it's just extra computation

Transformers with memory can abstract recursively in theory. https://arxiv.org/abs/2301.04589 https://arxiv.org/abs/2301.04589. If it's just extra computation that's the kick then simply more tokens will suffice. You can even even implement a computation token if you wish. https://arxiv.org/abs/2310.02226 https://arxiv.org/abs/2310.02226 Frankly, the ability to truly abstract recursively is by no means necessary either. Humans can't actually do this. Either in every day practice (you will get bored, you will lose interest) or in theory (flawed memory, finite memory, you will die). Limited recursive abilities =/ trivial recursive abilities. The current state of the art is easily general enough by actual testable definitions/baselines.