So one thing I wonder about is how well "smoothness" of the translated text reflects accuracy of the translation. What I mean is, we already know several reall

So one thing I wonder about is how well "smoothness" of the translated text reflects accuracy of the translation. What I mean is, we already know several really impressive examples of how language models from recurrent neural networks can generate eerily natural random texts (e.g. this blogpost http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/) So even if we just trained it on an English corpus and fed random numbers into it, it would still output smooth and natural-sounding English sentences. Of course, here it is actually translating, but I wonder how often it will be "overconfident", i.e. generate a plausible-sounding sentence which doesn't at all correspond to the input text. Unlike a human translator, it won't say "sorry I'm not sure what that means".