Thanks! The goal was to make local-only translation usable. I think cloud translation is still pretty valuable in a lot of cases since the model for one single

Thanks! The goal was to make local-only translation usable. I think cloud translation is still pretty valuable in a lot of cases since the model for one single direction translation is ~100MB. In addition to having more language options without a large download cloud translations let you use more specialized models for example French to Spanish. I just have a model to and from English for each language and any other translations have to "pivot" through English. For cloud translations you can also use one model with multiple input and output languages which gives you better quality translation between languages that don't have as much data available and lets you support direct translation between a large number of languages. Here's a talk where Google explains how they do this for Google Translate: https://youtu.be/nR74lBO5M3s?t=1682 https://youtu.be/nR74lBO5M3s?t=1682. You could do this locally but it would have its own set of challanges for getting the right model for the languages you want to translate. It would be possible, and probably not to hard to benchmark these translations (the standard method for automated testing is a BLEU score) but haven't bothered so far. I use Stanza for sentence boundary detection, SentencePiece for tokenization, and OpenNMT for translations which all seem to be about the best available open source. I think the most interesting thing about the underlying translation is that I use Stanza for sentence boundary detection which lets it deal with languages that don't use periods with one set of tools: https://forum.opennmt.net/t/sentence-boundary-detection-for-non-european-languages/3941 https://forum.opennmt.net/t/sentence-boundary-detection-for-.... An OpenNMT demo script I based my training script on claims a 28.0 BLEU score, which is pretty good, so that would probably be a reasonable estimate at least for language pairs with lots of high quality data available: https://github.com/OpenNMT/OpenNMT-tf/tree/230118d72b878760570418a8b36b8d02982e7b18/scripts/wmt https://github.com/OpenNMT/OpenNMT-tf/tree/230118d72b8787605....