Try it! It's simple enough to instruct it to work step by step, showing it's work (as a type of memory - same way we'd use pencil and paper), so that each stri

Try it! It's simple enough to instruct it to work step by step, showing it's work (as a type of memory - same way we'd use pencil and paper), so that each string transformation breaks down into a sequence of single replacement steps that it shows (input, rule applied, output). Tokenization is not the issue. Just give it the string as a space separated list of letters such as "A B B A X Y". The problem is getting it to work reliably, and for longer strings. The best I could do with Claude Haiku was to get it to apply three replacement rules successfully, then it just made a stupid mistake. LLMs are just not built for tasks like this. A lack of short-term memory is one obvious issue, but seems almost impossible to mitigate with step-by-step instruction.