They reference this paper on Byte Pair Encodings http://www.aclweb.org/anthology/P16-1162 http://www.aclweb.org/anthology/P16-1162 they tokenize text and then

They reference this paper on Byte Pair Encodings http://www.aclweb.org/anthology/P16-1162 http://www.aclweb.org/anthology/P16-1162 they tokenize text and then learn an embedding for those tokens using (in their case) both the source and target lanuage. This presumably captures regularities for languages that are not very different and has the benefit of having a small dictionary.