4 ms·

History PhD Student Reaches out to Cryptographers to Help Break a Civil War Code

115 points by justine 15y ago

lignuist 15y ago
Since he was using very uncommon symbols and the cipher is old, it's not so unlikely, that the author used a simple letter replacement table. Something like: a -> #, b -> *, c -> =, ... and so on. Unfortunately the handwriting is a bit messy and hard to read, but I'd try to make a frequency table for every symbol and compare that to a frequency table of letters in the english language: http://en.wikipedia.org/wiki/Letter_frequency http://en.wikipedia.org/wiki/Letter_frequency Now try to replace the most frequent symbol with e, the second most frequent symbol with a or t and so on. Try some variations and look, if it makes any sense.
- Locke1689 15y ago
  If those pictures are the only coded messages letter frequency may not have enough ciphertext to run on reliably.
  - lignuist 15y ago
    True, but that's why I'd try some variations.
- betawolf33 15y ago
  A simple substitution cipher is of course a good guess, but I suspect a homophonic one is more likely if the author knew anything about cryptography and thought it might come under attack. It's simple enough to come up with a number of homophones to complicate frequency analysis.
  - lignuist 15y ago
    > I suspect a homophonic one is more likely if the author knew anything about cryptography How likely is it, that he had a deeper knowledge of cryptography in the 1860ies? But of course, if he had, it's still likely, that he used something more sophisticated. http://en.wikipedia.org/wiki/History_of_cryptography#Cryptography_from_1800_to_World_War_II http://en.wikipedia.org/wiki/History_of_cryptography#Cryptog... Given the fact, that "12345" is still a common password in 2012, I would at least give it a shot. ;)
    - wowzer 15y ago
      I highly agree with linguist on this point. It's very unlikely a homophonic substitution was used here. It's more complicated than you'd think. A great read on the subject can be found here: http://www.amazon.com/Code-Book-Science-Secrecy-Cryptography/dp/0385495323 http://www.amazon.com/Code-Book-Science-Secrecy-Cryptography... if anyone's interested in learning more about cryptography. Very interesting.
    - betawolf33 15y ago
      Well, I'm just speculating idly of course, and yes, there's no sound reason to start suspecting anything more sophisticated than a substitution cipher yet, but cryptography was reasonably topical then. People used ciphers to conceal messages they had to send via telegraph (having to watch someone read and tap out a private message of yours mustn't be pleasant) and to encrypt messages put in personals sections of newspapers. Things like Poe's contest in Philadelphia and the Beale Papers apparently caused a public stir. The reason I suggest a homophonic cipher (or similar) rather than something like the much more secure Vignere in use for telegraph messages (only recently broken for the time) is because the Vignere system is more complicated, and requires more working out than I would expect someone writing a private message to tolerate. Homophonic systems, on the other hand, are fairly easy to invent and remember on a personal basis, and can offer some security against amateur analysis. Though the technique for solving them was known, they could still prove robust - the partially homophonic cipher of Lous XIV was still unbroken at this point, despite being over a century old. I'm very much an amateur (though I have read that book) but it doesn't seem that ridiculous to suggest a homophonic cipher. At any rate, It's only something to consider if it turns out to be something more complex than a simple substitution cipher.
- larrydag 15y ago
  Intresting thought. You could also use the number frequency of letters found in the rest of the journal passages that are not cryptic. That perhaps may be a better indicator for the symbols as it's the writers own tendencies.
deleted 15y ago
[deleted]
deleted 15y ago
[deleted]
indubitably 15y ago
It's astonishing to me how little understanding people have of Unicode.
stickfigure 15y ago
It will take a major feat of cryptanalysis just to decipher the guy's handwriting, let alone the code parts. Thanks for reminding me why I work with computers.
freeak 15y ago
Is this what is needed?: 1. Assign ascii to the symbols in the code 2. Transcribe the code to ascii 3. Solve the code in ascii using techniques from Snyder and Barzilay
- deserted 15y ago
  Yes.
madao 15y ago
perhaps its a form of shorthand http://upload.wikimedia.org/wikipedia/commons/thumb/e/ef/Eclectic_shorthand_by_cross.png/200px-Eclectic_shorthand_by_cross.png http://upload.wikimedia.org/wikipedia/commons/thumb/e/ef/Ecl...
deserted 15y ago
Forwarded to a cryptography Professor and some of his students : )
efsavage 15y ago
I've somehow been conditioned to think of these Civil War diaries and letters as full of flowery prose and beautiful handwriting. It's actually amazing to me to see someone writing plainly and with handwriting almost as bad as my own.
jerfelix 15y ago
Here's the first four images... ready for parsing by your program: :::: image 1, left: s-tac-toe equals minus-dot seven ex comma slash-slash-backslash gamma capital-l lower-j ex slash :::: image 1, right, downward capital-t capital-i equals four parallel-lines slash-slash-backslash equals-slash comma s-tac-toe slash-slash-backslash slash-slash-backslash-backslash comma :::: image 1, left, upper capital-l divided-by capital-i comma equivalent comma lower-j comma capital-f slash-slash-backslash capital-i squared-capital-n zee slash-slash-backslash-backslash comma divided-by slash-slash comma slash-backslash-backslash slash-slash minus-dot slash-slash-backslash :::: image 2 plus-dot plus leaning-heart upsidedown-t minus vertical-line ex comma leaning-heart capital-m capital-i capital-a minus three-peaks comma vertical-line crap capital-b close-bracket plus script-j script-s lower-d comma u-bar three-peaks capital-i :::: image 3 capital-l backslash divided-by c-slash-slash capital-i comma capital-i equivalent divided-by slash-slash-backslash-backslash comma capital-i minus-dot three-horizontal-two-vertical ex comma equals zee capital-l l-in-l 11-over-1 comma y-slash-slash slash-slash-backslash-backslash comma capital-i divided-by comma minus-lower-dot c-omega slash-slash slash-i 11-over-1 slash-slash square-c equals capital-l equivalent slash-slash comma capital-l slash-slash slash comma l-on-l plus slash-backslash-backslash slash-slash-backslash-backslash comma 1-slash-1 11-over-1 capital-z comma capital-i equals ex comma j divided-by c-slash-slash slash-slash capital-l divided-by slash-slash ex comma :::: image 4 (repeats image 2) plus-dot plus leaning-heart upsidedown-t minus vertical-line ex comma leaning-heart capital-m capital-i capital-a minus three-peaks comma vertical-line crap capital-b close-bracket plus script-j script-s lower-d comma u-bar three-peaks capital-i
- jerfelix 15y ago
  Here's a frequency chart of the first 3 images: (23, 'comma') (10, 'capital-i') (8, 'slash-slash') (7, 'divided-by') (7, 'capital-l') (6, 'ex') (5, 'slash-slash-backslash-backslash') (5, 'slash-slash-backslash') (5, 'equals') (3, 'plus') (3, 'minus-dot') (3, 'equivalent') (3, '11-over-1') (2, 'zee') (2, 'vertical-line') (2, 'three-peaks') (2, 'slash-backslash-backslash') (2, 'slash') (2, 's-tac-toe') (2, 'minus') (2, 'lower-j') (2, 'leaning-heart') (2, 'c-slash-slash') (1, 'y-slash-slash') (1, 'upsidedown-t') (1, 'u-bar') (1, 'three-horizontal-two-vertical') (1, 'squared-capital-n') (1, 'square-c') (1, 'slash-i') (1, 'seven') (1, 'script-s') (1, 'script-j') (1, 'plus-dot') (1, 'parallel-lines') (1, 'minus-lower-dot') (1, 'lower-d') (1, 'l-on-l') (1, 'l-in-l') (1, 'j') (1, 'gamma') (1, 'four') (1, 'equals-slash') (1, 'crap') (1, 'close-bracket') (1, 'capital-z') (1, 'capital-t') (1, 'capital-m') (1, 'capital-f') (1, 'capital-b') (1, 'capital-a') (1, 'c-omega') (1, 'backslash') (1, '1-slash-1')

115 points by justine 15y ago

Since he was using very uncommon symbols and the cipher is old, it's not so unlikely, that the author used a simple letter replacement table. Something like: a -> #, b -> *, c -> =, ... and so on. Unfortunately the handwriting is a bit messy and hard to read, but I'd try to make a frequency table for every symbol and compare that to a frequency table of letters in the english language: http://en.wikipedia.org/wiki/Letter_frequency http://en.wikipedia.org/wiki/Letter_frequency Now try to replace the most frequent symbol with e, the second most frequent symbol with a or t and so on. Try some variations and look, if it makes any sense.

Locke1689 15y ago
If those pictures are the only coded messages letter frequency may not have enough ciphertext to run on reliably.
- lignuist 15y ago
  True, but that's why I'd try some variations.
betawolf33 15y ago
A simple substitution cipher is of course a good guess, but I suspect a homophonic one is more likely if the author knew anything about cryptography and thought it might come under attack. It's simple enough to come up with a number of homophones to complicate frequency analysis.
- lignuist 15y ago
  > I suspect a homophonic one is more likely if the author knew anything about cryptography How likely is it, that he had a deeper knowledge of cryptography in the 1860ies? But of course, if he had, it's still likely, that he used something more sophisticated. http://en.wikipedia.org/wiki/History_of_cryptography#Cryptography_from_1800_to_World_War_II http://en.wikipedia.org/wiki/History_of_cryptography#Cryptog... Given the fact, that "12345" is still a common password in 2012, I would at least give it a shot. ;)
  - wowzer 15y ago
    I highly agree with linguist on this point. It's very unlikely a homophonic substitution was used here. It's more complicated than you'd think. A great read on the subject can be found here: http://www.amazon.com/Code-Book-Science-Secrecy-Cryptography/dp/0385495323 http://www.amazon.com/Code-Book-Science-Secrecy-Cryptography... if anyone's interested in learning more about cryptography. Very interesting.
  - betawolf33 15y ago
    Well, I'm just speculating idly of course, and yes, there's no sound reason to start suspecting anything more sophisticated than a substitution cipher yet, but cryptography was reasonably topical then. People used ciphers to conceal messages they had to send via telegraph (having to watch someone read and tap out a private message of yours mustn't be pleasant) and to encrypt messages put in personals sections of newspapers. Things like Poe's contest in Philadelphia and the Beale Papers apparently caused a public stir. The reason I suggest a homophonic cipher (or similar) rather than something like the much more secure Vignere in use for telegraph messages (only recently broken for the time) is because the Vignere system is more complicated, and requires more working out than I would expect someone writing a private message to tolerate. Homophonic systems, on the other hand, are fairly easy to invent and remember on a personal basis, and can offer some security against amateur analysis. Though the technique for solving them was known, they could still prove robust - the partially homophonic cipher of Lous XIV was still unbroken at this point, despite being over a century old. I'm very much an amateur (though I have read that book) but it doesn't seem that ridiculous to suggest a homophonic cipher. At any rate, It's only something to consider if it turns out to be something more complex than a simple substitution cipher.
larrydag 15y ago
Intresting thought. You could also use the number frequency of letters found in the rest of the journal passages that are not cryptic. That perhaps may be a better indicator for the symbols as it's the writers own tendencies.

deleted 15y ago

[deleted]

deleted 15y ago

[deleted]

indubitably 15y ago

It's astonishing to me how little understanding people have of Unicode.

stickfigure 15y ago

It will take a major feat of cryptanalysis just to decipher the guy's handwriting, let alone the code parts. Thanks for reminding me why I work with computers.

freeak 15y ago

Is this what is needed?: 1. Assign ascii to the symbols in the code 2. Transcribe the code to ascii 3. Solve the code in ascii using techniques from Snyder and Barzilay

deserted 15y ago
Yes.

madao 15y ago

perhaps its a form of shorthand http://upload.wikimedia.org/wikipedia/commons/thumb/e/ef/Eclectic_shorthand_by_cross.png/200px-Eclectic_shorthand_by_cross.png http://upload.wikimedia.org/wikipedia/commons/thumb/e/ef/Ecl...

deserted 15y ago

Forwarded to a cryptography Professor and some of his students : )

efsavage 15y ago

I've somehow been conditioned to think of these Civil War diaries and letters as full of flowery prose and beautiful handwriting. It's actually amazing to me to see someone writing plainly and with handwriting almost as bad as my own.

jerfelix 15y ago

Here's the first four images... ready for parsing by your program: :::: image 1, left: s-tac-toe equals minus-dot seven ex comma slash-slash-backslash gamma capital-l lower-j ex slash :::: image 1, right, downward capital-t capital-i equals four parallel-lines slash-slash-backslash equals-slash comma s-tac-toe slash-slash-backslash slash-slash-backslash-backslash comma :::: image 1, left, upper capital-l divided-by capital-i comma equivalent comma lower-j comma capital-f slash-slash-backslash capital-i squared-capital-n zee slash-slash-backslash-backslash comma divided-by slash-slash comma slash-backslash-backslash slash-slash minus-dot slash-slash-backslash :::: image 2 plus-dot plus leaning-heart upsidedown-t minus vertical-line ex comma leaning-heart capital-m capital-i capital-a minus three-peaks comma vertical-line crap capital-b close-bracket plus script-j script-s lower-d comma u-bar three-peaks capital-i :::: image 3 capital-l backslash divided-by c-slash-slash capital-i comma capital-i equivalent divided-by slash-slash-backslash-backslash comma capital-i minus-dot three-horizontal-two-vertical ex comma equals zee capital-l l-in-l 11-over-1 comma y-slash-slash slash-slash-backslash-backslash comma capital-i divided-by comma minus-lower-dot c-omega slash-slash slash-i 11-over-1 slash-slash square-c equals capital-l equivalent slash-slash comma capital-l slash-slash slash comma l-on-l plus slash-backslash-backslash slash-slash-backslash-backslash comma 1-slash-1 11-over-1 capital-z comma capital-i equals ex comma j divided-by c-slash-slash slash-slash capital-l divided-by slash-slash ex comma :::: image 4 (repeats image 2) plus-dot plus leaning-heart upsidedown-t minus vertical-line ex comma leaning-heart capital-m capital-i capital-a minus three-peaks comma vertical-line crap capital-b close-bracket plus script-j script-s lower-d comma u-bar three-peaks capital-i

jerfelix 15y ago
Here's a frequency chart of the first 3 images: (23, 'comma') (10, 'capital-i') (8, 'slash-slash') (7, 'divided-by') (7, 'capital-l') (6, 'ex') (5, 'slash-slash-backslash-backslash') (5, 'slash-slash-backslash') (5, 'equals') (3, 'plus') (3, 'minus-dot') (3, 'equivalent') (3, '11-over-1') (2, 'zee') (2, 'vertical-line') (2, 'three-peaks') (2, 'slash-backslash-backslash') (2, 'slash') (2, 's-tac-toe') (2, 'minus') (2, 'lower-j') (2, 'leaning-heart') (2, 'c-slash-slash') (1, 'y-slash-slash') (1, 'upsidedown-t') (1, 'u-bar') (1, 'three-horizontal-two-vertical') (1, 'squared-capital-n') (1, 'square-c') (1, 'slash-i') (1, 'seven') (1, 'script-s') (1, 'script-j') (1, 'plus-dot') (1, 'parallel-lines') (1, 'minus-lower-dot') (1, 'lower-d') (1, 'l-on-l') (1, 'l-in-l') (1, 'j') (1, 'gamma') (1, 'four') (1, 'equals-slash') (1, 'crap') (1, 'close-bracket') (1, 'capital-z') (1, 'capital-t') (1, 'capital-m') (1, 'capital-f') (1, 'capital-b') (1, 'capital-a') (1, 'c-omega') (1, 'backslash') (1, '1-slash-1')