That's not what the paper says. It says it possible to construct weights that when run through inference will perform Turing machine -equivalent operations on p

That's not what the paper says. It says it possible to construct weights that when run through inference will perform Turing machine -equivalent operations on prompts that are specifically made for that purpose. That does not mean weights derived from a pile of books will do such a thing.