If people are interested in the underlying architecture of Google's Neural Machine Translation (GNMT) system, I wrote an article that builds it up piece by piec

If people are interested in the underlying architecture of Google's Neural Machine Translation (GNMT) system, I wrote an article that builds it up piece by piece. While it's intended for people who are likely to implement GNMT or similar architectures, the article is descriptive enough that it should be possible to follow along even if you're not well versed in deep learning. http://smerity.com/articles/2016/google_nmt_arch.html http://smerity.com/articles/2016/google_nmt_arch.html The GNMT architecture is used almost as is for the zero-shot MT experiments. We're likely to see the GNMT architecture used extensively by Google for a variety of projects as they spent a deal of time and effort ensuring it is scalable to quite large datasets. Training a neural machine translation system with a single language pair is difficult - training it with multiple, especially all using the same set of weights, is insanely challenging! As an example, the GNMT architecture was used as the basis of "Generating Long and Diverse Responses with Neural Conversation Models", which trains on the entirety of Reddit (1.7 billion messages) as well as other various datasets. https://openreview.net/forum?id=HJDdiT9gl https://openreview.net/forum?id=HJDdiT9gl