This is simply not true, the pause token insertion is not situation dependent. If the model could say "please run me again", that would be groundbreaking and it

This is simply not true, the pause token insertion is not situation dependent. If the model could say "please run me again", that would be groundbreaking and it has been looked into. The problem is how do you train such a model? It would have to be reinforcement learning or something. Your set of allowed target sequences is infinite. I don't understand. You have an S by S matrix. You increase S, the matrix grows quadratically. There is no way around this.