4 ms·
Neat! If I'm reading this right (smarter people please correct me if I'm wrong), the process they used is: 1- Train a system that translates language A sentenc
by mabbo 9y ago
Neat! If I'm reading this right (smarter people please correct me if I'm wrong), the process they used is:
1- Train a system that translates language A sentences into a representation space, and can translate back from that space.
2- Train a second system that does the same, but with language B, onto the same representation space.
3- Train an adversarial system that tries to look at the representation space and identify which language the sentence came from, retraining the language translation systems to try to fool that recognizer. Retrain the models to try to not be recognized by this third system.
The best way to 'hide' from the recognizer is to have a very similar distribution, to make it use similar points in the representation space for the same concepts.
Brilliant stuff.
- OscarCunningham 9y agoI wonder if this relies on the sentences of the corpora referencing similar concepts. I'd be interested to see if it worked on a corpus of modern English and a corpus of classical Latin or Greek.
- nl 9y agoTemporal drift of concepts is definitely a thing - there's a good paper from Stanford's NLP group showing it in word embedding for English. The interesting thing is that this model might automatically correct for it (although of course it couldn't translate things for which no concept exists).
- visarga 9y ago> the model starts with an unsupervised naive translation model obtained by making word by word translation of sentences using a parallel dictionary learned in an unsupervised way So they DID have a parallel dictionary at first. How they learned it is another problem.
- matheist 9y agoI believe they also add a loss term corresponding to translating A to B using last iteration's model, then translating that B-sentence back to A and comparing to the original sentence.