> Do you rearchitect the code upon new research breakthroughs? Great question. So we now have implementations of Transformer in both the PyTorch and TensorFlow

> Do you rearchitect the code upon new research breakthroughs? Great question. So we now have implementations of Transformer in both the PyTorch and TensorFlow version of the library. It did require some rearchitecting, particularly for inference, but at heart the model is basically a sequence-to-sequence model. I wrote a blog post describing the process of implementing it here: http://nlp.seas.harvard.edu/2018/04/03/attention.html http://nlp.seas.harvard.edu/2018/04/03/attention.html . > Why use this over the Transformer model in tensor2tensor? OpenNMT is a bit more accessible to tensor2tensor which is a very powerful library but requires buying into a heavyweight framework. For instance OpenNMT uses plain text files, whereas tensor2tensor manages the entire data pipeline. I also personally find TensorFlow to be difficult for developing new research code.