6 ms·

The problem would be to define what is considered a successful outcome, measurable by an automated system.

analogmemory 8y ago
Wouldn't a successful outcome just be recognizable sentences?
tobinfricke 8y ago
Indeed, even if you decode the supposedly "cracked" Beale letter, you get nearly complete gibberish: ihaie depos otedi nthec opntt olBed oorta boupf ourmi lesfr ombul ordsi nanep caiat ionor iault sipfe stbel owthe surla csoft hhgto undth sfotl owing artic issbe aongi ngjoi otltt othep artfe swhos lnamf sateg iieti nnumb erthr ffhtt ewith https://nibot-lab.livejournal.com/tag/beale%20ciphers https://nibot-lab.livejournal.com/tag/beale%20ciphers It's almost certainly a hoax.
- DoctorOetker 8y ago
  I also believe it was a hoax, but this is not a good argument for it being a hoax: The perpetrator would expect buyers of the pamphlet to try reproduce the decoding of the second ciphertext, so presumably there were versions of the declaration that correctly formed the key. For the edits (compared to original declaration) see also: https://en.wikipedia.org/wiki/Beale_ciphers#Deciphered_message https://en.wikipedia.org/wiki/Beale_ciphers#Deciphered_messa... I would like to see a python (or other) script with the most important observations reproduced (such as the alphabet like sequence)
  - tobinfricke 8y ago
    > The perpetrator would expect buyers of the pamphlet to try reproduce the decoding of the second ciphertext, so presumably there were versions of the declaration that correctly formed the key. The version of the DoI that was used to decode ciphertext #2 was included with the pamphlet. It does not produce an error-free decoding. > I would like to see a python (or other) script with the most important observations reproduced (such as the alphabet like sequence) The link in the post to which you are replying contains, at the bottom, all three ciphertexts, the key text, and a C++ program that performs the decoding.
    - DoctorOetker 8y ago
      Regarding the similar distribution you pointed out in the plot, I was considering that perhaps the cipher numbers were ordered in a gridlike fashion on the original, such that perhaps vertical or diagonal reading, or more exact positioning would reveal extra information. So I was browsing the NSA pdf files "The Beale Papers" from archive.org, and apperently according to these files theres even different versions of the CIPHERTEXTS! It is unclear to me why there would be different ciphertexts floating around, possibly: 1) the pamphlet author published multiple versions 2) after analysis, later authors "fixed" the ciphertexts as opposed to describing the encryption errors in the decoding mechanism 3) flat out disinformation by: the publisher (how does the anonymous author defend the true ciphertext? would other publishers at the time dare to publish a second version of the cipher? how does this author prove he is the same anonymous person?), disinformation by treasure hunters (you can recognnize your own manipulation, but this confuses everyone else) Do we know what happened with the publisher? did it merge with others, and is it still in existence under a new name? Do they perhaps have any early original (from the box, or original manuscript from the anonymous person, etc...)
    - DoctorOetker 8y ago
      You probably know of this page, but I think this has the clearest explanation for decoding errors: http://rogergrambihler.tripod.com/BealeHoax.htm http://rogergrambihler.tripod.com/BealeHoax.htm although he doesn't notice his own mismatcc of "hith" instead of "with" (I haven't checked/reproduced his mapping in code, I intend to implement his decoding with quirks by adapting your .c file, if you wish I can send it over when done)
    - DoctorOetker 8y ago
      The alphabetic sequences in the first "cipher" may simply be key material: you can speed up your work by constructing a look-up table that maps each letter of the alphabet to a list of suitable numbers for words starting with that letter. While encoding is a work in progress this is a perfectly good tool to use, but it is dangerous to explicitly write the letter i.e. "A: 107, 204, ... B: 48, 87, ..." (I made up the numbers) because if the lookup table is found with the ciphertext, the attacker no longer needs to find the correct book (DoI)! An alternative to actually writing down the letters on the same piece of paper is to have a second strip of paper, to be held above/below the numbers, in an aligned fashion, so that one paper has "A [whitespaces] B [whitespaces] C ..." and the second has some numbers corresponding to the letters. (if you look at things like "indenture" the concept of aligning paper was a commonly used trick for verification etc purpouses) The grass is always greener on the other side: 1) whenever the encoder was working without lookup table, he labouriously had to traverse the DoI lookinng for a suitable word, wishing he had a lookup table 2) whenever the encoder retried constructing a good lookup table, he (erroneously?) felt he was wasting his time constructing a lookup table instead of encoding the text this could explain the restarting runs of the alphabet (with letters recurring i.e. aaaabbbccc..., and also explains why the number distribution is flatter compared to N and E as in "Where are the N and E characters" at http://rogergrambihler.tripod.com/BealeHoax.htm http://rogergrambihler.tripod.com/BealeHoax.htm )
- mickronome 8y ago
  It's almost certainly a hoax, but any book substitution cipher is very sensitive to errors both in counting, and in exactly which book. From that perspective, if you remove the spaces, it's almost intelligeble even to a non native speaker. Observe: ihaiedeposotedinthecopnttolBedoortaboupfourmilesfrombulordsinanepcaiationoriaultsipfestbelowthesurlacsofthhgtoundthsfotlowingarticissbeaongingjoiotlttothepartfeswhoslnamfsategiietinnumberthr ffhttewith i haie deposoted in the copntt ol Bedoort aboup four miles from bulords in an epcaiation or iault sip fest below the surlacs of thh gtound ths fotlowing articiss beaonging joiotltto the partfes whosl namfs ate giiet in number thrffhtte with I have deposited in the county of Bedoort (Bedford?) about for miles from bulords in an epcaiation or vault/fault six feet below the surface of the ground. Supposedly epcaiation is excavation, that one I didn't get. Some of the latter parts I also have trouble reading without guessing a lot. Like 'joiotltto' and 'thrffhtte'.
  - killaken2000 8y ago
    Also in the past there was no standardized way of writing words so its possible that some spellings are off by today's standards. I remember when the spellings of words became standardized.
  - paulie_a 8y ago
    > Supposedly epcaiation is excavation I buy it, spelling in that area at that time wasn't exactly top knotch. I know that is stereotypical, but come on.
- DoctorOetker 8y ago
  Your frequency plot is very peculiar and almost unexplainable: 1) while it can be explained if the author was intelligent enough to understand letter frequency cryptanalysis, but how widespread would that idea have been in his day and age? 2) even though the symbol frequencies match up, the text seems gibberish, which for now I can only explain as follows: The author of the pamphlet somehow obtained the 3 original ciphertexts and sincerely believed in their authenticity. (even if his source may or may not have scammed him). He identified the DOI as the correct key for ciphertext 2 (this insight too may have come from the potential source). Trivially the same key is tried on ciphertext 1. The author notices the quasi alphabet and otherwise gibberish. Unable to solve the puzzle he notes that if he cuts up decipherable ciphertext 2 in say 7 letter chunks, they clearly form random parts of english phrases. Similarily he cuts up ciphertext 1 in say 7 letter chunks (but leaves the alphabet part intact), and publishes it, in the hope that someone will find a "weird convincing but failed decoding", such that he only has to restore the ordering of the letters (since he knows how he reordered the 7 letter chunks). I.e. the pamphlet author might sincerely believe in the authenticity of the originnal ciphertext, but is trying to scam his audience into solving the puzzle for him!
- DoctorOetker 8y ago
  I fixed your decryptor, the code is here, feel free to update no credit needed: ```cpp #include <iostream> #include <string> #include <vector> #include <fstream> using namespace std; char quirkdecode(int i, vector<string> dict); int main(int argc, char argv) { // Load the dictionary vector<string> dict; ifstream dictstream("doi.txt"); string str; while (dictstream >> str) dict.push_back(str); int i, j=0; while (cin >> i) { j ++; cout << quirkdecode(i,dict); if (j%5 == 0) cout << " "; if (j%75 == 0) cout << endl; //*/ } cout << endl; return 0; } char quirkdecode(int i, vector<string> dict){ int delta=0; if (i==155) return 'a';// Beale Paper DOI has extra 'a' in "institute a new government" if (i>155) delta-=1; // Compensate for missing 'a' in doi.txt if (i>=242) delta+=1; // 241 decrypts correct 246 not so delta +1 @ [241-245] if (i>=480) delta+=10; // 466 decrypts correct 485 not so delta +10 @ [267-285] known misnumbering if (i>=510) delta-=1; // TODO presicely ocate if (i>=621) delta+=1; // 620 decrypts correct 643 is not so delta +1 @ [621-643] if (i>=667) delta+=1; // 666 decrypts correct 807 is not so delta +1 @ [667-807] ? double check ?? // Hardcoded Jokers: NOTE 811 and 1005 are also the highest used codes, so it DOES seem to be intended as real if (i==811) return 'y' ; // fundamentallY if (i==1005) return 'x' ; // seXes i+=delta; if (i<dict.size()) return dict.at(i-1)[0]; else return '!'; } ``` which gives: ``` ihave depos itedi nthec ounty ofBed forda boutf ourmi lesfr ombuf ordsi nanex cavat ionor vault sixfe etbel owthe surfa ceoft hegro undth efoll owing artic lesbe longi ngjoi ntlyt othep artie swhos ename sareg iveni nnumb erthr eeher ewith thefi rstde posit consi stcdo ftenh undre dandf ourte enpou ndsof golda ndthi rtyei ghthu ndred andtw elvep ounds ofsil verde posit ednov eight eenni netee nthes econd Wasma dedec eight eentw entyo neand consi stedo fnine teenh undre dands evenp ounds ofgol dandt welve hundr edand eight yeigh tofsi lvera lsoje welso btain edins tloui sinex chang etosa vetra nspor tatio nandv alued atthi rteen rhous anddo llars theab oveis secur elypa ckedi niron potsw ithir oncov ersth evaul tisro ughly lined withs tonea ndthe vesse lsres tonso lidst onean darec overe dwith other spape rnumb erone descr ibest hcexa ctloc ality ofthe varlt sotha tnodi fficu ltywi llbeh adinf indin git ```
ComplexSystems 8y ago
There are, I dunno, maybe 1,000,000 words in the English language. With a book cipher, almost all books will produce gibberish words not on the list. If you find one for which almost all words are magically on that list of 1,000,000 words, you win. I get the appeal of the notion that some second grader could stumble on the winning book and claim the prize, but this article is like halfway between cryptographic fantasy and reality. Google suggests there are only 130,000,000 books in existence, which would be more than easy enough to brute force if we had access to all of them. Furthermore, there is a clear success criterion. The limiting factor is that not all books have been digitized, but if the book is relatively well known, it should be. I'm surprised someone at Google hasn't just put this whole thing to bed by now.
- tobinfricke 8y ago
  See my post below. Even using the supposedly-correct key, the decoding produces near-gibberish. It would certainly not pass a test that requires the decoded plaintext to consist of dictionary words.

Wouldn't a successful outcome just be recognizable sentences?

Indeed, even if you decode the supposedly "cracked" Beale letter, you get nearly complete gibberish: ihaie depos otedi nthec opntt olBed oorta boupf ourmi lesfr ombul ordsi nanep caiat ionor iault sipfe stbel owthe surla csoft hhgto undth sfotl owing artic issbe aongi ngjoi otltt othep artfe swhos lnamf sateg iieti nnumb erthr ffhtt ewith https://nibot-lab.livejournal.com/tag/beale%20ciphers https://nibot-lab.livejournal.com/tag/beale%20ciphers It's almost certainly a hoax.

DoctorOetker 8y ago
I also believe it was a hoax, but this is not a good argument for it being a hoax: The perpetrator would expect buyers of the pamphlet to try reproduce the decoding of the second ciphertext, so presumably there were versions of the declaration that correctly formed the key. For the edits (compared to original declaration) see also: https://en.wikipedia.org/wiki/Beale_ciphers#Deciphered_message https://en.wikipedia.org/wiki/Beale_ciphers#Deciphered_messa... I would like to see a python (or other) script with the most important observations reproduced (such as the alphabet like sequence)
- tobinfricke 8y ago
  > The perpetrator would expect buyers of the pamphlet to try reproduce the decoding of the second ciphertext, so presumably there were versions of the declaration that correctly formed the key. The version of the DoI that was used to decode ciphertext #2 was included with the pamphlet. It does not produce an error-free decoding. > I would like to see a python (or other) script with the most important observations reproduced (such as the alphabet like sequence) The link in the post to which you are replying contains, at the bottom, all three ciphertexts, the key text, and a C++ program that performs the decoding.
  - DoctorOetker 8y ago
    Regarding the similar distribution you pointed out in the plot, I was considering that perhaps the cipher numbers were ordered in a gridlike fashion on the original, such that perhaps vertical or diagonal reading, or more exact positioning would reveal extra information. So I was browsing the NSA pdf files "The Beale Papers" from archive.org, and apperently according to these files theres even different versions of the CIPHERTEXTS! It is unclear to me why there would be different ciphertexts floating around, possibly: 1) the pamphlet author published multiple versions 2) after analysis, later authors "fixed" the ciphertexts as opposed to describing the encryption errors in the decoding mechanism 3) flat out disinformation by: the publisher (how does the anonymous author defend the true ciphertext? would other publishers at the time dare to publish a second version of the cipher? how does this author prove he is the same anonymous person?), disinformation by treasure hunters (you can recognnize your own manipulation, but this confuses everyone else) Do we know what happened with the publisher? did it merge with others, and is it still in existence under a new name? Do they perhaps have any early original (from the box, or original manuscript from the anonymous person, etc...)
  - DoctorOetker 8y ago
    You probably know of this page, but I think this has the clearest explanation for decoding errors: http://rogergrambihler.tripod.com/BealeHoax.htm http://rogergrambihler.tripod.com/BealeHoax.htm although he doesn't notice his own mismatcc of "hith" instead of "with" (I haven't checked/reproduced his mapping in code, I intend to implement his decoding with quirks by adapting your .c file, if you wish I can send it over when done)
  - DoctorOetker 8y ago
    The alphabetic sequences in the first "cipher" may simply be key material: you can speed up your work by constructing a look-up table that maps each letter of the alphabet to a list of suitable numbers for words starting with that letter. While encoding is a work in progress this is a perfectly good tool to use, but it is dangerous to explicitly write the letter i.e. "A: 107, 204, ... B: 48, 87, ..." (I made up the numbers) because if the lookup table is found with the ciphertext, the attacker no longer needs to find the correct book (DoI)! An alternative to actually writing down the letters on the same piece of paper is to have a second strip of paper, to be held above/below the numbers, in an aligned fashion, so that one paper has "A [whitespaces] B [whitespaces] C ..." and the second has some numbers corresponding to the letters. (if you look at things like "indenture" the concept of aligning paper was a commonly used trick for verification etc purpouses) The grass is always greener on the other side: 1) whenever the encoder was working without lookup table, he labouriously had to traverse the DoI lookinng for a suitable word, wishing he had a lookup table 2) whenever the encoder retried constructing a good lookup table, he (erroneously?) felt he was wasting his time constructing a lookup table instead of encoding the text this could explain the restarting runs of the alphabet (with letters recurring i.e. aaaabbbccc..., and also explains why the number distribution is flatter compared to N and E as in "Where are the N and E characters" at http://rogergrambihler.tripod.com/BealeHoax.htm http://rogergrambihler.tripod.com/BealeHoax.htm )
mickronome 8y ago
It's almost certainly a hoax, but any book substitution cipher is very sensitive to errors both in counting, and in exactly which book. From that perspective, if you remove the spaces, it's almost intelligeble even to a non native speaker. Observe: ihaiedeposotedinthecopnttolBedoortaboupfourmilesfrombulordsinanepcaiationoriaultsipfestbelowthesurlacsofthhgtoundthsfotlowingarticissbeaongingjoiotlttothepartfeswhoslnamfsategiietinnumberthr ffhttewith i haie deposoted in the copntt ol Bedoort aboup four miles from bulords in an epcaiation or iault sip fest below the surlacs of thh gtound ths fotlowing articiss beaonging joiotltto the partfes whosl namfs ate giiet in number thrffhtte with I have deposited in the county of Bedoort (Bedford?) about for miles from bulords in an epcaiation or vault/fault six feet below the surface of the ground. Supposedly epcaiation is excavation, that one I didn't get. Some of the latter parts I also have trouble reading without guessing a lot. Like 'joiotltto' and 'thrffhtte'.
- killaken2000 8y ago
  Also in the past there was no standardized way of writing words so its possible that some spellings are off by today's standards. I remember when the spellings of words became standardized.
- paulie_a 8y ago
  > Supposedly epcaiation is excavation I buy it, spelling in that area at that time wasn't exactly top knotch. I know that is stereotypical, but come on.
DoctorOetker 8y ago
Your frequency plot is very peculiar and almost unexplainable: 1) while it can be explained if the author was intelligent enough to understand letter frequency cryptanalysis, but how widespread would that idea have been in his day and age? 2) even though the symbol frequencies match up, the text seems gibberish, which for now I can only explain as follows: The author of the pamphlet somehow obtained the 3 original ciphertexts and sincerely believed in their authenticity. (even if his source may or may not have scammed him). He identified the DOI as the correct key for ciphertext 2 (this insight too may have come from the potential source). Trivially the same key is tried on ciphertext 1. The author notices the quasi alphabet and otherwise gibberish. Unable to solve the puzzle he notes that if he cuts up decipherable ciphertext 2 in say 7 letter chunks, they clearly form random parts of english phrases. Similarily he cuts up ciphertext 1 in say 7 letter chunks (but leaves the alphabet part intact), and publishes it, in the hope that someone will find a "weird convincing but failed decoding", such that he only has to restore the ordering of the letters (since he knows how he reordered the 7 letter chunks). I.e. the pamphlet author might sincerely believe in the authenticity of the originnal ciphertext, but is trying to scam his audience into solving the puzzle for him!
DoctorOetker 8y ago
I fixed your decryptor, the code is here, feel free to update no credit needed: ```cpp #include <iostream> #include <string> #include <vector> #include <fstream> using namespace std; char quirkdecode(int i, vector<string> dict); int main(int argc, char argv) { // Load the dictionary vector<string> dict; ifstream dictstream("doi.txt"); string str; while (dictstream >> str) dict.push_back(str); int i, j=0; while (cin >> i) { j ++; cout << quirkdecode(i,dict); if (j%5 == 0) cout << " "; if (j%75 == 0) cout << endl; //*/ } cout << endl; return 0; } char quirkdecode(int i, vector<string> dict){ int delta=0; if (i==155) return 'a';// Beale Paper DOI has extra 'a' in "institute a new government" if (i>155) delta-=1; // Compensate for missing 'a' in doi.txt if (i>=242) delta+=1; // 241 decrypts correct 246 not so delta +1 @ [241-245] if (i>=480) delta+=10; // 466 decrypts correct 485 not so delta +10 @ [267-285] known misnumbering if (i>=510) delta-=1; // TODO presicely ocate if (i>=621) delta+=1; // 620 decrypts correct 643 is not so delta +1 @ [621-643] if (i>=667) delta+=1; // 666 decrypts correct 807 is not so delta +1 @ [667-807] ? double check ?? // Hardcoded Jokers: NOTE 811 and 1005 are also the highest used codes, so it DOES seem to be intended as real if (i==811) return 'y' ; // fundamentallY if (i==1005) return 'x' ; // seXes i+=delta; if (i<dict.size()) return dict.at(i-1)[0]; else return '!'; } ``` which gives: ``` ihave depos itedi nthec ounty ofBed forda boutf ourmi lesfr ombuf ordsi nanex cavat ionor vault sixfe etbel owthe surfa ceoft hegro undth efoll owing artic lesbe longi ngjoi ntlyt othep artie swhos ename sareg iveni nnumb erthr eeher ewith thefi rstde posit consi stcdo ftenh undre dandf ourte enpou ndsof golda ndthi rtyei ghthu ndred andtw elvep ounds ofsil verde posit ednov eight eenni netee nthes econd Wasma dedec eight eentw entyo neand consi stedo fnine teenh undre dands evenp ounds ofgol dandt welve hundr edand eight yeigh tofsi lvera lsoje welso btain edins tloui sinex chang etosa vetra nspor tatio nandv alued atthi rteen rhous anddo llars theab oveis secur elypa ckedi niron potsw ithir oncov ersth evaul tisro ughly lined withs tonea ndthe vesse lsres tonso lidst onean darec overe dwith other spape rnumb erone descr ibest hcexa ctloc ality ofthe varlt sotha tnodi fficu ltywi llbeh adinf indin git ```

ComplexSystems 8y ago

There are, I dunno, maybe 1,000,000 words in the English language. With a book cipher, almost all books will produce gibberish words not on the list. If you find one for which almost all words are magically on that list of 1,000,000 words, you win. I get the appeal of the notion that some second grader could stumble on the winning book and claim the prize, but this article is like halfway between cryptographic fantasy and reality. Google suggests there are only 130,000,000 books in existence, which would be more than easy enough to brute force if we had access to all of them. Furthermore, there is a clear success criterion. The limiting factor is that not all books have been digitized, but if the book is relatively well known, it should be. I'm surprised someone at Google hasn't just put this whole thing to bed by now.

tobinfricke 8y ago
See my post below. Even using the supposedly-correct key, the decoding produces near-gibberish. It would certainly not pass a test that requires the decoded plaintext to consist of dictionary words.