4 ms·
What's interesting about neural machine translation is that the core model is completely language pair independent. So we roughly use the same code for Russian-
by srush 10y ago
What's interesting about neural machine translation is that the core model is completely language pair independent. So we roughly use the same code for Russian-English, English-Russian, and Chinese-German. That being said the errors in Russian are quite different than those made in other languages due to case endings. For instance for a similar size data set there are often 5x more unique Russian words than in English.
But if you want to get involved more generally our gitter is http://gitter.im/OpenNMT http://gitter.im/OpenNMT and our forum is at http://forum.opennmt.net http://forum.opennmt.net.
- kmicklas 10y agoThis seems like an argument for a character based rather than word based network. It just so happens that English and Chinese, the two languages which have the most machine learning research, are relatively analytic, with a low morpheme per word ratio. But many world languages have a much higher ratio (Russian wouldn't even rank that high!) and acquiring training data covering all unique "words" is essentially impossible.
- srush 10y agoI'm glad you mentioned this. There is a lot of interest these days in character-based machine translation, including several papers in review at ICLR. The current practical consensus (at least in OpenNMT) is that character-only models are not really worth the efficiency loss. A simple compromise is to use Byte-Pair Encoding as a preprocessing step in morphologically rich languages and allow the model to produce sub-word chunks. This is implemented in OpenNMT as a preprocessing option (see http://opennmt.net/Advanced http://opennmt.net/Advanced).