3 ms·

I've recently carried out a through statistical analysis of the text and concluded that it's almost certainly meaningless, and have figured out how the text cou

I've recently carried out a through statistical analysis of the text and concluded that it's almost certainly meaningless, and have figured out how the text could have been generated. I'll start a new thread ... (https://news.ycombinator.com/item?id=14091216 https://news.ycombinator.com/item?id=14091216)

Nursie 10y ago
On a brief scan that looks hideously complicated and more than a little like extracting a property of some text and the being surprised that when you replay you get something similar. Can you say that your technique, when applied to any and all natural languages, absolutely does not do something similar? I'm not sure many people are going to buy it.
- DonaldFisk 10y ago
  I had that thought myself. Briefly, my theory is that the different parts of the text were generated using state transition tables. The actual weights on the transitions are arbitrary, and can only be determined by reverse engineering the original manuscript. Only one (physical) table needs to be generated at first, but the values of the transition weights do need to be changed as the scribe proceeds through the manuscript. Could it be used for English? I doubt it. The morphology of English and other Indo-European languages is too complex to for a reasonably-sized state machine to realistically generate. Maybe for something more regular such as Pinyin Mandarin or Yale Cantonese BUT the output wouldn't fool anyone who knows those languages, and a linguist who had never seen them before would very quickly become suspicious - no discernable word order, common words repeated several times in a row.
  - Nursie 10y ago
    Well it's certainly another interesting hypothesis, I wouldn't be ready to call it quite yet though!
  - mcbits 10y ago
    Some people with schizophrenia invent their own languages and alphabets, and their speech can seem randomly generated in severe cases. Or an alchemist might have been exposed to chemicals that caused schizophrenic symptoms. I wonder if the statistics would be similar.
    - DonaldFisk 10y ago
      While it's still worth testing that, my understanding is that it's very difficult for anyone to think up values indistinguishable from random, and this appears to be what we're dealing with here. What I'm proposing is that the words we see are the output of a random process, but that their order appears to be random, in the statistical sense. However, everyone who's studied the text agrees that the glyph distribution within words conforms to a well-defined syntax, and there are significant differences between different parts of the text. My theory explains these. Of course, a well-designed cipher will appear to be random, and the input to the text generation process appears to be just that, but the problem is that there's no way of subsequently decrypting the text, as information (in the mathematical sense, as well as the everyday sense) would be lost.

Nursie 10y ago

On a brief scan that looks hideously complicated and more than a little like extracting a property of some text and the being surprised that when you replay you get something similar. Can you say that your technique, when applied to any and all natural languages, absolutely does not do something similar? I'm not sure many people are going to buy it.

DonaldFisk 10y ago
I had that thought myself. Briefly, my theory is that the different parts of the text were generated using state transition tables. The actual weights on the transitions are arbitrary, and can only be determined by reverse engineering the original manuscript. Only one (physical) table needs to be generated at first, but the values of the transition weights do need to be changed as the scribe proceeds through the manuscript. Could it be used for English? I doubt it. The morphology of English and other Indo-European languages is too complex to for a reasonably-sized state machine to realistically generate. Maybe for something more regular such as Pinyin Mandarin or Yale Cantonese BUT the output wouldn't fool anyone who knows those languages, and a linguist who had never seen them before would very quickly become suspicious - no discernable word order, common words repeated several times in a row.
- Nursie 10y ago
  Well it's certainly another interesting hypothesis, I wouldn't be ready to call it quite yet though!
- mcbits 10y ago
  Some people with schizophrenia invent their own languages and alphabets, and their speech can seem randomly generated in severe cases. Or an alchemist might have been exposed to chemicals that caused schizophrenic symptoms. I wonder if the statistics would be similar.
  - DonaldFisk 10y ago
    While it's still worth testing that, my understanding is that it's very difficult for anyone to think up values indistinguishable from random, and this appears to be what we're dealing with here. What I'm proposing is that the words we see are the output of a random process, but that their order appears to be random, in the statistical sense. However, everyone who's studied the text agrees that the glyph distribution within words conforms to a well-defined syntax, and there are significant differences between different parts of the text. My theory explains these. Of course, a well-designed cipher will appear to be random, and the input to the text generation process appears to be just that, but the problem is that there's no way of subsequently decrypting the text, as information (in the mathematical sense, as well as the everyday sense) would be lost.