3 ms·

I just read the original paper (http://aclweb.org/anthology-new/W/W11/W11-1202.pdf http://aclweb.org/anthology-new/W/W11/W11-1202.pdf). As far as I can tell, t

by kleiba 15y ago

I just read the original paper (http://aclweb.org/anthology-new/W/W11/W11-1202.pdf http://aclweb.org/anthology-new/W/W11/W11-1202.pdf). As far as I can tell, the article by theregister is complete nonsense: the researchers suspected a simple substitution cipher and used statistical analysis of the distribution of (co-)occurrences of the text symbols. Once they conjectured that the source language was German, they were able to use letter frequency information from modern German texts to derive likely mappings between the symbols and letters from the German alphabet. This has nothing to do with machine translation. MT does not typically operate on the level of letters. The only time the researchers mention machine translation in their paper is on page 6, where they piped the partly deciphered text into a German-to-English translation software to see where their mapping was incorrect. The remarkable fact is that apparently none of the researches speaks German, so they used such software to check if it could make sense of the sequences of letters deciphered so far. They might as well have used an old-fashioned dictionary to achieve the same result. On page 7 they talk about finally talking to native speakers of German (quite late in the endavour I think), to help them with the remaining problems. The register article gives the impression that the researchers sort of pasted a transcripted version of the original document into an MT system which then magically produced a "translation" into German autonomously. That is not at all what happened.

maaku 15y ago
It's even less remarkable if you've ever done that sort of cryptanalysis. I was assigned a similar problem in an entry-level college course, and the solution ended up being something like a dozen actual lines of code. Seriously. Just for kicks I generalized it automatically detect the source language and adapt accordingly. The article is interesting for its historical aspect, but that's about it.
pvg 15y ago
The cypher is a substitution one but not a 'simple' one since multiple symbols map to single plaintext letters and some symbols map to n-grams common in German. The analysis is related to language identification, rather than machine translation as a whole - even if the researchers guessed the plaintext language correctly to begin with. It doesn't seem particularly remarkable none of them spoke German, they certainly spoke languages close enough to it to recognize their initial results were German-looking enough - at which point they enlisted the help of German speakers. They did, in fact, just use a dictionary at one point. The analysis is more sophisticated than simply guessing the source language and the fact it's a substitution-type cypher and the techniques on the n-gram level are arguably not entirely unlike what Google uses for MT, but on the word and phrase level. You're right that drawing a direct connection between them is somewhat tendentious.
- kleiba 15y ago
  Your first point is fair enough, it is not the "simplest" substitution cipher one could imagine (one-to-one mapping), but I would say it's still a fairly simple mapping. The reason I mentioned that none of the researchers speaks German is remarkable in two senses. First, it is the only reason they had to use machine translation in their approach at all! It was only for their testing the quality of their intermediate results, and not - as theregister suggests - a central part of the deciphering. Second, the fact that you don't have to speak the target language gives some credit to their methodology, because to some degree there's hope that the complete algorithm could be automated.
  - tripzilch 15y ago
    "Simple substitution cipher" is a cryptographic jargon, that means substituting single characters for other characters or symbols, independent of their context or position. See http://en.wikipedia.org/wiki/Substitution_cipher#Simple_substitution http://en.wikipedia.org/wiki/Substitution_cipher#Simple_subs...
    - kleiba 15y ago
      Oh, nice, I didn't know that!
DanBC 15y ago
> the article by theregister is complete nonsense The Register is awful. Linking to them for this article is especially bad because the sources are so easily available. The error about machine translation occurs because members of the team doing this work are recognised experts in machine translation.

by kleiba 15y ago

maaku 15y ago

It's even less remarkable if you've ever done that sort of cryptanalysis. I was assigned a similar problem in an entry-level college course, and the solution ended up being something like a dozen actual lines of code. Seriously. Just for kicks I generalized it automatically detect the source language and adapt accordingly. The article is interesting for its historical aspect, but that's about it.

pvg 15y ago

The cypher is a substitution one but not a 'simple' one since multiple symbols map to single plaintext letters and some symbols map to n-grams common in German. The analysis is related to language identification, rather than machine translation as a whole - even if the researchers guessed the plaintext language correctly to begin with. It doesn't seem particularly remarkable none of them spoke German, they certainly spoke languages close enough to it to recognize their initial results were German-looking enough - at which point they enlisted the help of German speakers. They did, in fact, just use a dictionary at one point. The analysis is more sophisticated than simply guessing the source language and the fact it's a substitution-type cypher and the techniques on the n-gram level are arguably not entirely unlike what Google uses for MT, but on the word and phrase level. You're right that drawing a direct connection between them is somewhat tendentious.

kleiba 15y ago
Your first point is fair enough, it is not the "simplest" substitution cipher one could imagine (one-to-one mapping), but I would say it's still a fairly simple mapping. The reason I mentioned that none of the researchers speaks German is remarkable in two senses. First, it is the only reason they had to use machine translation in their approach at all! It was only for their testing the quality of their intermediate results, and not - as theregister suggests - a central part of the deciphering. Second, the fact that you don't have to speak the target language gives some credit to their methodology, because to some degree there's hope that the complete algorithm could be automated.
- tripzilch 15y ago
  "Simple substitution cipher" is a cryptographic jargon, that means substituting single characters for other characters or symbols, independent of their context or position. See http://en.wikipedia.org/wiki/Substitution_cipher#Simple_substitution http://en.wikipedia.org/wiki/Substitution_cipher#Simple_subs...
  - kleiba 15y ago
    Oh, nice, I didn't know that!

DanBC 15y ago

> the article by theregister is complete nonsense The Register is awful. Linking to them for this article is especially bad because the sources are so easily available. The error about machine translation occurs because members of the team doing this work are recognised experts in machine translation.