10 ms·

Does Machine Translation Affect International Trade?

80 points by cracker_jacks 6y ago

dwohnitmok 6y ago
Machine translation feels like one of the quiet victories of the current crop of machine learning products. It's getting to be within spitting distance of being completely adequate (though not excellent) for translating long-form articles, which is amazing considering where things stood ten years ago.
- Baeocystin 6y ago
  I find that machine translation is getting better at producing what reads as grammatically correct text, but that it still misses a lot of what is said, even when the idioms and expressions used are common. Often what is produced is grammatical, but opposite/orthogonal of the initial intent. In a sense, I feel that this is an even worse situation, because the word-salad signal that it isn't an accurate-meaning translation isn't as strong.
  - xbmcuser 6y ago
    I think the problem is when direct translation is done. Netflix paper using simplify then translating to me seems to be the better approach when you want to convey the meaning rather than just translating the exact words https://arxiv.org/abs/2005.11197 https://arxiv.org/abs/2005.11197
  - userbinator 6y ago
    That's what MT based on neural networks will do: their output can look great, but the correspondence with the input can be highly inaccurate. On the other hand, traditional methods based on syntactic analysis tend to yield less grammatically correct and more literal translations, but the literalness can also result in better accuracy.
    - MiroF 6y ago
      Mm... I think you'd be hard pressed at this point to find any advantage that traditional methods have over deep methods. Maybe this was true 4-6 years ago, but no longer.
      - _-___________-_ 6y ago
        Depends what you optimise for. Neural translation fairly consistently produces incorrect results that read like they could be correct, for me. If your goal is "text that makes sense" they seem to be doing great. If your goal is "text that conveys the same meaning as the source" they're universally rubbish, at least for the language pairs I need (mostly english <-> east Asian languages).
        MiroF 6y ago
        > english <-> east Asian languages I'm curious what the domain is and what languages you're talking about, it's hard for me to evaluate your claims otherwise. I would buy it for English -> Tibetan for instance.
        _-___________-_ 6y ago
        Mostly Chinese, Japanese, and Vietnamese, and mostly business and lightly-technical text. My observation of the failure rate (if you define failure as "meaning has been lost") is well above 80% with Google Translate for these languages and this type of text. It's literally unusable because you have no confidence that the translation has not completely reversed or garbled the meaning of the text, and the failure mode is particularly unpleasant because you may be unaware that it has failed to translate correctly, since it produces text that appears to have meaning - just the wrong meaning.
        numpad0 6y ago
        Kinda funny when people say "East Asia" it mostly means Japan and its isolationism, with China mixed in just to change the tone a bit. I don't find it offensive, just kind of funny/interesting.
- klodolph 6y ago
  For some language pairs, maybe. The problem is that as you translate between more distant languages, you find that something which is contextual in one language is textual in the other, and vice versa. The neural networks are great at faking it, and they'll make up text that looks completely reasonable all day long. If that means inventing subjects and choosing random verb tenses, that's what they'll do. If you pick closely-related languages, you might be able to get a near perfect translation just by putting each word through a dictionary, and the neural networks can disambiguate things and smooth over the problems in the output by looking at local context. If you pick distantly-related languages, you might routinely need context from paragraphs away, or context which isn't present in the text at all but relies on some kind of human experience.
  - MiroF 6y ago
    > more distant languages, low resource translation is definitely still bad, you're right. > near perfect translation just by putting each word through a dictionary, and the neural networks can disambiguate things and smooth over the problems in the output Funnily enough, this is not a terrible explanation of how the current sort of approaches to low-resource MT work.
    - thaumasiotes 6y ago
      >> more distant languages, > low resource translation is definitely still bad, you're right. No, you've misunderstood. Languages that are closely related to each other are fairly easy to translate between. Languages that aren't related are not, regardless of whether you have a large corpus to work with. English - Mandarin is not an example of "low resource translation".
      - MiroF 6y ago
        > Languages that aren't related are not, regardless of whether you have a large corpus to work with The size of the corpus makes a huge difference. I don't see how can both be true that a. there is this massive failure of translation engines on these language pairs, b. human translators were unable to differentiate between human translated chinese-english sentences and machine translated sentences. It's not incredible - but it's also not awful or unusable.
- _-___________-_ 6y ago
  I really have to disagree. Machine translation is utterly terrible, and appears to be optimising for the wrong thing. All the progress seems to be in making it produce output that makes sense, rather than producing output that conveys the same meaning as the source document. Almost every time I've tried to use machine translation for something serious, the meaning of the text has been garbled or reversed. This is not "adequate" unless your only goal is to produce something that looks like it could be correct, without regard for whether it is.
- danielscrubs 6y ago
  What now? How? It’s getting better by the day but translation is one of the places where it has done more damage than good. It teaches language students incorrect grammar and hinder students from changing their thought pattern. “- Milk tea or regular tea? -Yes!” Yes is the proper response in some languages. You can either change the language or your mindset. And it’s not like these things are one offs, all languages are basically a bunch of quirks. No doubt it’s the way of the future but I’d wait for the researchers to improve translation a lot.
userbinator 6y ago
I thought the "large digital platform" would be AliExpress (which from experience I know does use MT extensively in seller-buyer communications), but it's actually referring to eBay. My experience with the former is that the MT is far from perfect, and even frequently causes much amusement, but definitely useful.
- dwohnitmok 6y ago
  > My experience with the former is that the MT is far from perfect, and even frequently causes much amusement, but definitely useful. Oh fascinating, do you have any idea of what things machine translation still consistently gets wrong?
  - _jahh 6y ago
    not op but i find idiom or proper nouns can mess things up.
    - Polylactic_acid 6y ago
      Another one is when one language uses a single word for two meanings which the other language has split. An example I cam across is when I translated the word "amber" meaning the orange color and the translation came out the other end as a word that purely means "fossilized sap"
    - 082349872349872 6y ago
      I generally learn proper nouns first (following news is my biggest use of MT) and I've found it often helps to replace them by placeholders such as X and Y when translating. As to sibling comment, IIRC Yandex translate has a nice feature where they'll provide a number of meanings for words when clicked upon in the source text. "Invisible, Insane"
  - klodolph 6y ago
    Oh, so much. Machine translation doesn't understand context and routinely fails at fairly basic translation tasks. Here's part of the Google translation of the front page of ja.wikipedia.org into English: > 非行少年の再非行の抑止や更生を目的としており、決定までの過程として、「非行事実を家庭裁判所に送致・通告 - 家庭裁判所調査官（以下「調査官」と略称する）等による調査 - 調査結果をふまえた審判 - 必要に応じて保護的措置あるいは保護処分を決定」という流れを経るのが通例である。 > And for the purpose of deterrence and rehabilitation of re-delinquency of juvenile delinquents, as the process of until a decision, " the fact delinquency to the family court -up Reel-notice - family court investigators (referred to hereinafter as" investigators "), or the like by the survey - survey It is customary to go through the flow of " judgment based on the result- decide protective measures or protective measures as necessary ". That's barely recognizable as English. The translation is hot garbage. Perhaps something you might use if you had an emergency. Just to point out here... the text on the front page of Wikipedia tends to be very neutral, matter-of-fact, and well written. There are no characters (like in novels), no use of slang, and nothing else that should be tricky for translation, and yet we get this garbage. It's a miracle that machine translation works at all, and when it does work, it is usually because you are translating between languages that are fairly similar to begin with. For example, English and Spanish.
peteretep 6y ago
I’ve got to believe Sichuan food would be more popular if 口水鸡 was less often translated as “saliva chicken” than “mouth-wateringly good chicken”
- mc32 6y ago
  Maybe "drool chicken" would be a compromise?
- hinoki 6y ago
  Google translates “口水鶏” as mouthwater chicken, which still lacks some appeal :)
- gibolt 6y ago
  Mouth liquid fowl sounds quite appetizing
- karmasimida 6y ago
  Well I, for one, have long believing that this mouth-watering chicken recipe isn't complete/authentic without the grandma spitting the last ingredient into the glorious final mix. And I understand Chinese pretty well.
- jjoonathan 6y ago
  Another one: the Chinese word for product/good, which is a bit more generic than its English counterpart and might be translated as "it/thing". One of the definitions under that broad umbrella is "baby" (analogous to "little thing" I suppose) -- and for the longest time, despite year after year of Google publishing advanced neural translation models, google translate reliably chose "baby" as the translation in the context of online commerce. I bought babies, got good deals on babies, and even tried to return a defective baby once, but it didn't work out. It looks like this has now been fixed, but it was good fun for a long time.
  - fireattack 6y ago
    I have no idea what word do you mean. 东西? 玩意?
pjfin123 6y ago
Shameless plug for my open source neural machine translation project. A Python library and GUI with pre-trained models that packages the files needed for machine translation and makes them usable through a simple interface. https://github.com/argosopentech/argos-translate https://github.com/argosopentech/argos-translate
- pabs3 6y ago
  And here is a non-ML open source machine translation project: https://www.apertium.org/ https://www.apertium.org/
  - pjfin123 6y ago
    Yes Apertium is great! If you go back in the commit logs my project actually started as a GUI for doing Apertium translations. Apertium does "rules-based" translations which works best for translating between very similar languages like Spanish-Catalan (it was initially funded by the Spanish government), and supports a number of languages. However, currently statistical and neural net approaches have better performance for most language pairs. My initial goal was to write the GUI for Apertium, release it on the Snap store, and then try to figure out OpenNMT. I had it working inside of a snap package but ran into an issue with uploading to the Snap store so I moved on: https://forum.snapcraft.io/t/unable-to-upload-to-snap-store-because-of-too-many-layouts/17595 https://forum.snapcraft.io/t/unable-to-upload-to-snap-store-...
    - pabs3 6y ago
      I think we need a machine translation library with multiple backends (Apertium, local ML based MT tools, the online ML based MT APIs) and then have toolkits, desktops, email clients, browsers and other things that encounter untranslated text use that library to convert that text into the user's language.
      - pjfin123 6y ago
        I agree I think especially for supporting a large number of languages your going to want to use online APIs so you don't need to store a lot of models locally. For common language pairs, or ones you use frequently it would be nice for privacy and offline use to be able to seamlessly do it locally.
jacobwilliamroy 6y ago
I think it's time to give up on machine translation. Let people do it. Maybe try and teach the computers how to do something we can't already do ourselves.
unhammer 6y ago
For low-resource languages, or anything of great linguistic distance, neural MT is still not there. That is to say, you may find some language pairs where it works OK, but there's a huge sea of pairs where all you get is gibberish – and still they publish these systems and let people use them to produce more gibberish which they put on their blogs and spam sites, making the non-English web into a corpus of dissociated press ramblings. Baidu recently added Saami to their system. They manage to make a Wikipedia article about wolverines into one about spreadsheets, depressed buttons and famous chickens: https://imgur.com/a/m91SbRw https://imgur.com/a/m91SbRw A rough translation of what Baidu gave: > No one is greater than zero. There are new things for spreadsheet[here Baidu used the English word for some reason] and all the things in it. > English is an active wolf, but the background as wolf indicates that the active button is depressed. > Too many places to install. Sweden was released from 1968 and July 1973. Probably number of times the alarm is to be reapeated within 39 minutes. 2010 is in total 66 chickens, but later a famous chicken and chickens 12th of march 2010, with a total of 54 chickens Apertium's translation is often stilted, but it gets the meaning across: Wolverines are the largest animals in the marten family, etc. Google Translate isn't half bad at Norwegian→English, so you can even combine them to get a sort of Saami→English: https://translate.google.com/#view=home&op=translate&sl=no&tl=en&text=Jerven%20(Gulo%20gulo)%20er%20hos%20m%C3%A5rfamilien%20med%20det%20st%C3%B8rste%20dyret.%20De%20andre%20store%20m%C3%A5rdyrene%20er%20Storoter%20og%20havoter.%20Jerven%20finnes%20spredt%20i%20cirkumpolare%20fjell%20og%20h%C3%B8yt%20landskap.%0A%0AP%C3%A5%20engelsk%20er%20det%20%C2%ABjerv%C2%BB%20wolverine%2C%20hvis%20bakgrunnen%20er%20wolvering%20som%20som%20verdien%20det%20betyr%20at%20%C2%ABhan%20oppf%C3%B8rer%20seg%20som%20ulv%C2%BB.%20%0A%0AN%C3%A5%20har%20mange%20land%20fredlyst%20jerven.%20Se%20Bern-konvensjonens%20%C2%ABLista%20II%C2%BB%2C%20som%20ble%20etablert%20av%201982.%20I%20Sverige%20ble%20geatki1968%20fredet%23%3Ai%20s%20og%20S%C3%B8r-Norge%20i%201973.%20Myndighetene%20har%20bestemt%20de%20minste%20tallets%20rovdyr%2C%20som%20skal%20v%C3%A6re%20i%20Norge.%20Jerven%20med%20hensyn%20til%20det%20er%20m%C3%A5lsetting%20at%20de%20skal%20v%C3%A6re%2039%20f%C3%A5%20i%20%C3%A5ret.%20I%202010%20er%20de%20for%20informasjonen%2066%20f%C3%A5%20ciiko%2C%20men%20etter%20jakta%20hvor%20det%20drepte%20ciiko%20og%20ungene%20i%2012%20hi%2C%20s%C3%A5%20var%20de%20v%C3%A5ren%202010%20i%2054%20hi%20unger https://translate.google.com/#view=home&op=translate&sl=no&t.... I tried Baidu again now with the same input, just to see what they would give into English, and now it's about old ports, finished volcanos and civic computers: https://fanyi.baidu.com/#sme/en/Geatki%20(Gulo%20gulo)%20lea%20stuorimus%20ealli%20neahtebearra%C5%A1is.%20Ear%C3%A1%20stuora%20neahteeallit%20leat%20Stuora%C4%8Deavrris%20ja%20meara%C4%8Deavrris.%20Geatki%20g%C3%A1vdno%20bie%C4%91gguid%20sirkumpol%C3%A1ra%20v%C3%A1riin%20ja%20alla%20eatnamiin.%0A%0AE%C5%8Bgelasgillii%20lea%20%C2%ABgeatki%C2%BB%20wolverine%2C%20man%20duog%C3%A1%C5%A1%20lea%20wolvering%20mii%20%C3%A1rvvusge%20mearkka%C5%A1a%20ahte%20%C2%ABl%C3%A1htte%20dego%20gumpe%C2%BB.%20%0A%0AD%C3%A1l%20leat%20ollu%20riikkat%20r%C3%A1f%C3%A1iduhtt%C3%A1n%20geatkki.%20Geah%C4%8Da%20Bernkonven%C5%A1uvnna%20%C2%ABListtu%20II%C2%BB%2C%20mii%20%C3%A1sahuvvui%201982%3As.%20Ruo%C5%A7as%20r%C3%A1f%C3%A1idahttojuvvui%20geatki1968%3As%20ja%20Lulli-Norggas%201973%3As.%20Eisev%C3%A1lddit%20leat%20mearridan%20unnimus%20logu%20boraspiriid%2C%20mat%20galget%20leat%20Norggas.%20Geatki%20d%C3%A1fus%20lea%20mihttomearri%20ahte%20galget%20leat%2039%20%C4%8Divgama%20jagis.%202010%3As%20leat%20dihtosis%2066%20%C4%8Divgi%20ciiko%2C%20muhto%20ma%C5%8B%C5%8Bel%20bivddu%20gos%20godde%20ciiko%20ja%20%C4%8Divggaid%2012%20biejus%2C%20de%20ledje%202010%20gi%C4%91a%2054%20biejus%20%C4%8Divggat https://fanyi.baidu.com/#sme/en/Geatki%20(Gulo%20gulo)%20lea.... (but at least "Berners Convention" is slightly close to the input …) It's amazing how low a bar big companies have for publishing ML models that just completely fail in the most absurd and spectacular ways, as if GM said "hey marketing department it looks like people want electric cars now, that seems easy just strap a battery on a chicken and hammer some wheels onto it and call it a day"

80 points by cracker_jacks 6y ago

Machine translation feels like one of the quiet victories of the current crop of machine learning products. It's getting to be within spitting distance of being completely adequate (though not excellent) for translating long-form articles, which is amazing considering where things stood ten years ago.

Baeocystin 6y ago
I find that machine translation is getting better at producing what reads as grammatically correct text, but that it still misses a lot of what is said, even when the idioms and expressions used are common. Often what is produced is grammatical, but opposite/orthogonal of the initial intent. In a sense, I feel that this is an even worse situation, because the word-salad signal that it isn't an accurate-meaning translation isn't as strong.
- xbmcuser 6y ago
  I think the problem is when direct translation is done. Netflix paper using simplify then translating to me seems to be the better approach when you want to convey the meaning rather than just translating the exact words https://arxiv.org/abs/2005.11197 https://arxiv.org/abs/2005.11197
- userbinator 6y ago
  That's what MT based on neural networks will do: their output can look great, but the correspondence with the input can be highly inaccurate. On the other hand, traditional methods based on syntactic analysis tend to yield less grammatically correct and more literal translations, but the literalness can also result in better accuracy.
  - MiroF 6y ago
    Mm... I think you'd be hard pressed at this point to find any advantage that traditional methods have over deep methods. Maybe this was true 4-6 years ago, but no longer.
    - _-___________-_ 6y ago
      Depends what you optimise for. Neural translation fairly consistently produces incorrect results that read like they could be correct, for me. If your goal is "text that makes sense" they seem to be doing great. If your goal is "text that conveys the same meaning as the source" they're universally rubbish, at least for the language pairs I need (mostly english <-> east Asian languages).
      - MiroF 6y ago
        > english <-> east Asian languages I'm curious what the domain is and what languages you're talking about, it's hard for me to evaluate your claims otherwise. I would buy it for English -> Tibetan for instance.
        _-___________-_ 6y ago
        Mostly Chinese, Japanese, and Vietnamese, and mostly business and lightly-technical text. My observation of the failure rate (if you define failure as "meaning has been lost") is well above 80% with Google Translate for these languages and this type of text. It's literally unusable because you have no confidence that the translation has not completely reversed or garbled the meaning of the text, and the failure mode is particularly unpleasant because you may be unaware that it has failed to translate correctly, since it produces text that appears to have meaning - just the wrong meaning.
        numpad0 6y ago
        Kinda funny when people say "East Asia" it mostly means Japan and its isolationism, with China mixed in just to change the tone a bit. I don't find it offensive, just kind of funny/interesting.
klodolph 6y ago
For some language pairs, maybe. The problem is that as you translate between more distant languages, you find that something which is contextual in one language is textual in the other, and vice versa. The neural networks are great at faking it, and they'll make up text that looks completely reasonable all day long. If that means inventing subjects and choosing random verb tenses, that's what they'll do. If you pick closely-related languages, you might be able to get a near perfect translation just by putting each word through a dictionary, and the neural networks can disambiguate things and smooth over the problems in the output by looking at local context. If you pick distantly-related languages, you might routinely need context from paragraphs away, or context which isn't present in the text at all but relies on some kind of human experience.
- MiroF 6y ago
  > more distant languages, low resource translation is definitely still bad, you're right. > near perfect translation just by putting each word through a dictionary, and the neural networks can disambiguate things and smooth over the problems in the output Funnily enough, this is not a terrible explanation of how the current sort of approaches to low-resource MT work.
  - thaumasiotes 6y ago
    >> more distant languages, > low resource translation is definitely still bad, you're right. No, you've misunderstood. Languages that are closely related to each other are fairly easy to translate between. Languages that aren't related are not, regardless of whether you have a large corpus to work with. English - Mandarin is not an example of "low resource translation".
    - MiroF 6y ago
      > Languages that aren't related are not, regardless of whether you have a large corpus to work with The size of the corpus makes a huge difference. I don't see how can both be true that a. there is this massive failure of translation engines on these language pairs, b. human translators were unable to differentiate between human translated chinese-english sentences and machine translated sentences. It's not incredible - but it's also not awful or unusable.
_-___________-_ 6y ago
I really have to disagree. Machine translation is utterly terrible, and appears to be optimising for the wrong thing. All the progress seems to be in making it produce output that makes sense, rather than producing output that conveys the same meaning as the source document. Almost every time I've tried to use machine translation for something serious, the meaning of the text has been garbled or reversed. This is not "adequate" unless your only goal is to produce something that looks like it could be correct, without regard for whether it is.
danielscrubs 6y ago
What now? How? It’s getting better by the day but translation is one of the places where it has done more damage than good. It teaches language students incorrect grammar and hinder students from changing their thought pattern. “- Milk tea or regular tea? -Yes!” Yes is the proper response in some languages. You can either change the language or your mindset. And it’s not like these things are one offs, all languages are basically a bunch of quirks. No doubt it’s the way of the future but I’d wait for the researchers to improve translation a lot.

userbinator 6y ago

I thought the "large digital platform" would be AliExpress (which from experience I know does use MT extensively in seller-buyer communications), but it's actually referring to eBay. My experience with the former is that the MT is far from perfect, and even frequently causes much amusement, but definitely useful.

dwohnitmok 6y ago
> My experience with the former is that the MT is far from perfect, and even frequently causes much amusement, but definitely useful. Oh fascinating, do you have any idea of what things machine translation still consistently gets wrong?
- _jahh 6y ago
  not op but i find idiom or proper nouns can mess things up.
  - Polylactic_acid 6y ago
    Another one is when one language uses a single word for two meanings which the other language has split. An example I cam across is when I translated the word "amber" meaning the orange color and the translation came out the other end as a word that purely means "fossilized sap"
  - 082349872349872 6y ago
    I generally learn proper nouns first (following news is my biggest use of MT) and I've found it often helps to replace them by placeholders such as X and Y when translating. As to sibling comment, IIRC Yandex translate has a nice feature where they'll provide a number of meanings for words when clicked upon in the source text. "Invisible, Insane"
- klodolph 6y ago
  Oh, so much. Machine translation doesn't understand context and routinely fails at fairly basic translation tasks. Here's part of the Google translation of the front page of ja.wikipedia.org into English: > 非行少年の再非行の抑止や更生を目的としており、決定までの過程として、「非行事実を家庭裁判所に送致・通告 - 家庭裁判所調査官（以下「調査官」と略称する）等による調査 - 調査結果をふまえた審判 - 必要に応じて保護的措置あるいは保護処分を決定」という流れを経るのが通例である。 > And for the purpose of deterrence and rehabilitation of re-delinquency of juvenile delinquents, as the process of until a decision, " the fact delinquency to the family court -up Reel-notice - family court investigators (referred to hereinafter as" investigators "), or the like by the survey - survey It is customary to go through the flow of " judgment based on the result- decide protective measures or protective measures as necessary ". That's barely recognizable as English. The translation is hot garbage. Perhaps something you might use if you had an emergency. Just to point out here... the text on the front page of Wikipedia tends to be very neutral, matter-of-fact, and well written. There are no characters (like in novels), no use of slang, and nothing else that should be tricky for translation, and yet we get this garbage. It's a miracle that machine translation works at all, and when it does work, it is usually because you are translating between languages that are fairly similar to begin with. For example, English and Spanish.

peteretep 6y ago

I’ve got to believe Sichuan food would be more popular if 口水鸡 was less often translated as “saliva chicken” than “mouth-wateringly good chicken”

mc32 6y ago
Maybe "drool chicken" would be a compromise?
hinoki 6y ago
Google translates “口水鶏” as mouthwater chicken, which still lacks some appeal :)
gibolt 6y ago
Mouth liquid fowl sounds quite appetizing
karmasimida 6y ago
Well I, for one, have long believing that this mouth-watering chicken recipe isn't complete/authentic without the grandma spitting the last ingredient into the glorious final mix. And I understand Chinese pretty well.
jjoonathan 6y ago
Another one: the Chinese word for product/good, which is a bit more generic than its English counterpart and might be translated as "it/thing". One of the definitions under that broad umbrella is "baby" (analogous to "little thing" I suppose) -- and for the longest time, despite year after year of Google publishing advanced neural translation models, google translate reliably chose "baby" as the translation in the context of online commerce. I bought babies, got good deals on babies, and even tried to return a defective baby once, but it didn't work out. It looks like this has now been fixed, but it was good fun for a long time.
- fireattack 6y ago
  I have no idea what word do you mean. 东西? 玩意?

pjfin123 6y ago

Shameless plug for my open source neural machine translation project. A Python library and GUI with pre-trained models that packages the files needed for machine translation and makes them usable through a simple interface. https://github.com/argosopentech/argos-translate https://github.com/argosopentech/argos-translate

pabs3 6y ago
And here is a non-ML open source machine translation project: https://www.apertium.org/ https://www.apertium.org/
- pjfin123 6y ago
  Yes Apertium is great! If you go back in the commit logs my project actually started as a GUI for doing Apertium translations. Apertium does "rules-based" translations which works best for translating between very similar languages like Spanish-Catalan (it was initially funded by the Spanish government), and supports a number of languages. However, currently statistical and neural net approaches have better performance for most language pairs. My initial goal was to write the GUI for Apertium, release it on the Snap store, and then try to figure out OpenNMT. I had it working inside of a snap package but ran into an issue with uploading to the Snap store so I moved on: https://forum.snapcraft.io/t/unable-to-upload-to-snap-store-because-of-too-many-layouts/17595 https://forum.snapcraft.io/t/unable-to-upload-to-snap-store-...
  - pabs3 6y ago
    I think we need a machine translation library with multiple backends (Apertium, local ML based MT tools, the online ML based MT APIs) and then have toolkits, desktops, email clients, browsers and other things that encounter untranslated text use that library to convert that text into the user's language.
    - pjfin123 6y ago
      I agree I think especially for supporting a large number of languages your going to want to use online APIs so you don't need to store a lot of models locally. For common language pairs, or ones you use frequently it would be nice for privacy and offline use to be able to seamlessly do it locally.

jacobwilliamroy 6y ago

I think it's time to give up on machine translation. Let people do it. Maybe try and teach the computers how to do something we can't already do ourselves.

unhammer 6y ago

For low-resource languages, or anything of great linguistic distance, neural MT is still not there. That is to say, you may find some language pairs where it works OK, but there's a huge sea of pairs where all you get is gibberish – and still they publish these systems and let people use them to produce more gibberish which they put on their blogs and spam sites, making the non-English web into a corpus of dissociated press ramblings. Baidu recently added Saami to their system. They manage to make a Wikipedia article about wolverines into one about spreadsheets, depressed buttons and famous chickens: https://imgur.com/a/m91SbRw https://imgur.com/a/m91SbRw A rough translation of what Baidu gave: > No one is greater than zero. There are new things for spreadsheet[here Baidu used the English word for some reason] and all the things in it. > English is an active wolf, but the background as wolf indicates that the active button is depressed. > Too many places to install. Sweden was released from 1968 and July 1973. Probably number of times the alarm is to be reapeated within 39 minutes. 2010 is in total 66 chickens, but later a famous chicken and chickens 12th of march 2010, with a total of 54 chickens Apertium's translation is often stilted, but it gets the meaning across: Wolverines are the largest animals in the marten family, etc. Google Translate isn't half bad at Norwegian→English, so you can even combine them to get a sort of Saami→English: https://translate.google.com/#view=home&op=translate&sl=no&tl=en&text=Jerven%20(Gulo%20gulo)%20er%20hos%20m%C3%A5rfamilien%20med%20det%20st%C3%B8rste%20dyret.%20De%20andre%20store%20m%C3%A5rdyrene%20er%20Storoter%20og%20havoter.%20Jerven%20finnes%20spredt%20i%20cirkumpolare%20fjell%20og%20h%C3%B8yt%20landskap.%0A%0AP%C3%A5%20engelsk%20er%20det%20%C2%ABjerv%C2%BB%20wolverine%2C%20hvis%20bakgrunnen%20er%20wolvering%20som%20som%20verdien%20det%20betyr%20at%20%C2%ABhan%20oppf%C3%B8rer%20seg%20som%20ulv%C2%BB.%20%0A%0AN%C3%A5%20har%20mange%20land%20fredlyst%20jerven.%20Se%20Bern-konvensjonens%20%C2%ABLista%20II%C2%BB%2C%20som%20ble%20etablert%20av%201982.%20I%20Sverige%20ble%20geatki1968%20fredet%23%3Ai%20s%20og%20S%C3%B8r-Norge%20i%201973.%20Myndighetene%20har%20bestemt%20de%20minste%20tallets%20rovdyr%2C%20som%20skal%20v%C3%A6re%20i%20Norge.%20Jerven%20med%20hensyn%20til%20det%20er%20m%C3%A5lsetting%20at%20de%20skal%20v%C3%A6re%2039%20f%C3%A5%20i%20%C3%A5ret.%20I%202010%20er%20de%20for%20informasjonen%2066%20f%C3%A5%20ciiko%2C%20men%20etter%20jakta%20hvor%20det%20drepte%20ciiko%20og%20ungene%20i%2012%20hi%2C%20s%C3%A5%20var%20de%20v%C3%A5ren%202010%20i%2054%20hi%20unger https://translate.google.com/#view=home&op=translate&sl=no&t.... I tried Baidu again now with the same input, just to see what they would give into English, and now it's about old ports, finished volcanos and civic computers: https://fanyi.baidu.com/#sme/en/Geatki%20(Gulo%20gulo)%20lea%20stuorimus%20ealli%20neahtebearra%C5%A1is.%20Ear%C3%A1%20stuora%20neahteeallit%20leat%20Stuora%C4%8Deavrris%20ja%20meara%C4%8Deavrris.%20Geatki%20g%C3%A1vdno%20bie%C4%91gguid%20sirkumpol%C3%A1ra%20v%C3%A1riin%20ja%20alla%20eatnamiin.%0A%0AE%C5%8Bgelasgillii%20lea%20%C2%ABgeatki%C2%BB%20wolverine%2C%20man%20duog%C3%A1%C5%A1%20lea%20wolvering%20mii%20%C3%A1rvvusge%20mearkka%C5%A1a%20ahte%20%C2%ABl%C3%A1htte%20dego%20gumpe%C2%BB.%20%0A%0AD%C3%A1l%20leat%20ollu%20riikkat%20r%C3%A1f%C3%A1iduhtt%C3%A1n%20geatkki.%20Geah%C4%8Da%20Bernkonven%C5%A1uvnna%20%C2%ABListtu%20II%C2%BB%2C%20mii%20%C3%A1sahuvvui%201982%3As.%20Ruo%C5%A7as%20r%C3%A1f%C3%A1idahttojuvvui%20geatki1968%3As%20ja%20Lulli-Norggas%201973%3As.%20Eisev%C3%A1lddit%20leat%20mearridan%20unnimus%20logu%20boraspiriid%2C%20mat%20galget%20leat%20Norggas.%20Geatki%20d%C3%A1fus%20lea%20mihttomearri%20ahte%20galget%20leat%2039%20%C4%8Divgama%20jagis.%202010%3As%20leat%20dihtosis%2066%20%C4%8Divgi%20ciiko%2C%20muhto%20ma%C5%8B%C5%8Bel%20bivddu%20gos%20godde%20ciiko%20ja%20%C4%8Divggaid%2012%20biejus%2C%20de%20ledje%202010%20gi%C4%91a%2054%20biejus%20%C4%8Divggat https://fanyi.baidu.com/#sme/en/Geatki%20(Gulo%20gulo)%20lea.... (but at least "Berners Convention" is slightly close to the input …) It's amazing how low a bar big companies have for publishing ML models that just completely fail in the most absurd and spectacular ways, as if GM said "hey marketing department it looks like people want electric cars now, that seems easy just strap a battery on a chicken and hammer some wheels onto it and call it a day"