I am extremely interested in this. The utter silence in the literature around this is a bit odd, frankly. Not LLMs specifically, but transformers more general

I am extremely interested in this. The utter silence in the literature around this is a bit odd, frankly. Not LLMs specifically, but transformers more generally, ought to be extremely good at this task. Another one I wonder about a lot is execution traces. Something that isn't mentioned often enough is that transformer models can do previous-token completion just as easily as next-token completion. So you can train the model on paired program/exectrace (training data is trivial to generate here -- just execute it on a CPU!) and then ask the model to work backwards from a desired machine-state you want to reach.