These sorts of models do well at generating text, whether it's code or language. That's what you're talking about. They can perform the "execution" to some degr

These sorts of models do well at generating text, whether it's code or language. That's what you're talking about. They can perform the "execution" to some degree too for natural language (e.g. the squad tasks). But Q&A and text generation are very different tasks. I haven't seen anyone apply a transformer to source code based Q&A tasks.