5 ms·

It looks useful but the lack of non-European languages makes me slightly suspicious, I wonder if their approach generalises to Arabic, Chinese, Japanese, etc.

by mantap 7y ago

needle0 7y ago
Perhaps I'm too jaded, but it comes off to me as copping out of the truly difficult ones!
- eyegor 7y ago
  If you know of a good source for large piles of docs that have been accurately/naturally translated eng/ch/jp, I'm sure deepl would be interested. As another poster pointed out, any deep learning NLP project boils down to quantity+quality of data. I'm assuming adversarial approaches don't work well in this context but I'm not very familiar with nlp research.
  - mantap 7y ago
    Right but that's kind of the essence of machine translation. You won't always have high quality parallel training data for all languages so you have to find a way to thrive with low quality data. Google has clearly made it their mission to solve that problem, and I'd say they've been rather successful.
yorwba 7y ago
It depends on which part of their approch you focus on. I'd expect their machine translation model to be sufficiently general to support basically any language given enough training data. Yes, the languages you listed have some edge cases, but so do languages they already support fine. For example, the lack of spaces separating words in Chinese and Japanese can be handled by the same word-piece segmentation they need for German compound words. The bigger problem is likely to be lack of training data. Unless they have a pile of cash to pay professional translators to produce a parallel corpus, the alternative is to scrape translations from the internet. Basically, crawl the same site multiple times with different Accept-Language headers and try to align the results. Crucially, this depends on an existing ecosystem of bilingual websites with high-quality human translations. According to DeepL's website they're a spin-off of Linguee, who provide a search service for exactly that kind of parallel data. So before DeepL starts supporting any given language pair, you should expect it to appear in Linguee first. https://en.wikipedia.org/wiki/Linguee https://en.wikipedia.org/wiki/Linguee Edit: It took me a while to figure out how to select a different language on https://linguee.com https://linguee.com (Their UI seems broken using mobile Firefox Preview.) Appending /english-chinese and /english-japanese to the URL shows that they already support those two, and the alignment of translations appears reasonable to me. No /english-arabic, though.
- azepoi 7y ago
  The European Union, the Swiss confederation, Belgium, Canada and other multilingual states, the European patent office and many international organization provide a huge corpus of professionally translated documents and reports for major European languages. Not so much for Japanese.

It looks useful but the lack of non-European languages makes me slightly suspicious, I wonder if their approach generalises to Arabic, Chinese, Japanese, etc.

by mantap 7y ago

It looks useful but the lack of non-European languages makes me slightly suspicious, I wonder if their approach generalises to Arabic, Chinese, Japanese, etc.

needle0 7y ago

Perhaps I'm too jaded, but it comes off to me as copping out of the truly difficult ones!

eyegor 7y ago
If you know of a good source for large piles of docs that have been accurately/naturally translated eng/ch/jp, I'm sure deepl would be interested. As another poster pointed out, any deep learning NLP project boils down to quantity+quality of data. I'm assuming adversarial approaches don't work well in this context but I'm not very familiar with nlp research.
- mantap 7y ago
  Right but that's kind of the essence of machine translation. You won't always have high quality parallel training data for all languages so you have to find a way to thrive with low quality data. Google has clearly made it their mission to solve that problem, and I'd say they've been rather successful.

yorwba 7y ago

It depends on which part of their approch you focus on. I'd expect their machine translation model to be sufficiently general to support basically any language given enough training data. Yes, the languages you listed have some edge cases, but so do languages they already support fine. For example, the lack of spaces separating words in Chinese and Japanese can be handled by the same word-piece segmentation they need for German compound words. The bigger problem is likely to be lack of training data. Unless they have a pile of cash to pay professional translators to produce a parallel corpus, the alternative is to scrape translations from the internet. Basically, crawl the same site multiple times with different Accept-Language headers and try to align the results. Crucially, this depends on an existing ecosystem of bilingual websites with high-quality human translations. According to DeepL's website they're a spin-off of Linguee, who provide a search service for exactly that kind of parallel data. So before DeepL starts supporting any given language pair, you should expect it to appear in Linguee first. https://en.wikipedia.org/wiki/Linguee https://en.wikipedia.org/wiki/Linguee Edit: It took me a while to figure out how to select a different language on https://linguee.com https://linguee.com (Their UI seems broken using mobile Firefox Preview.) Appending /english-chinese and /english-japanese to the URL shows that they already support those two, and the alignment of translations appears reasonable to me. No /english-arabic, though.

azepoi 7y ago
The European Union, the Swiss confederation, Belgium, Canada and other multilingual states, the European patent office and many international organization provide a huge corpus of professionally translated documents and reports for major European languages. Not so much for Japanese.