I am not convinced. All models are trained with teacher forcing, thus the model never sees invalid sequences during training. It's only during inference that th

I am not convinced. All models are trained with teacher forcing, thus the model never sees invalid sequences during training. It's only during inference that this difference appears, as shown by the experiment of selfies without valency cobstrainnts. To me, it seems like this is an artifact caused by details of how sampling from a model is done, rather than something about the models or training procedures.