3 ms·

Everyone has their own way of stumbling through to a breakthrough, where suddenly things that had been confusing and complex suddenly seem clear, beautiful, and

Everyone has their own way of stumbling through to a breakthrough, where suddenly things that had been confusing and complex suddenly seem clear, beautiful, and intuitive. Here are a couple of thoughts on Godel's incompleteness theorem that helped me get there. First, a description of the idea; consider the following two statements: "There exists a formula with Godel number M that has the property that neither the formula nor its negation has a proof." "Oh, by the way. The Godel number of the above formula is M." "M" in the above is an actual number. In the first statement, one has an arithmetic expression (i.e., "3 + 4*5 + 12^100000 + ...") that is short but that evaluates to a really large number. After developing the idea of mapping formulas and proofs of first-order logic to integers, Godel needed to use his new tool to come up with some way to express self reference. (The formula above has to have an embedded arithmetic expression "M" that unwraps and and evaluates to the Godel number of the entire formula.) Godel devised what we would today recognize as exactly the Y combinator, expressed in first order arithmetic. This was a shocking realization when it dawned on me, and it enabled me to gain an insight to the magnificent subtlety of Godel's mind. I am personally comfortable with lisp, functions as first-class objects, lambda calculus, etc., as is certainly the case for many Hacker News readers. So, at least for me, the above connection helped an awful lot to really understand the heart of Godel's insight.

pyinstallwoes 3y ago
Take this with the most non-serious mode: In the methods used by Gematria and various arts, there is a similar mechanism at play is there not? Transforming words into quasi-hash coordinate space that is computable and thus able to see what is related or equal?
- gregfjohnson 3y ago
  Fascinating observation, thanks. Relevant to your comment but diverging farther from the original post: Coincidentally, I am taking a beginner's Kabbalah course right now. I wonder if some of the writers of scripture actually used steganography or other word encodings such as the ones you allude to intentionally. One obvious example of word play at the meta level might be Psalm 119.
  - pyinstallwoes 3y ago
    Yeah, along those lines I’ve joked that the Bible admits it’s written by a LLM: “in the beginning was the Word and the Word was with God and the Word was God” - that rings like a recursive declaration statement with Genesis and other similar statements.
- empath-nirvana 3y ago
  The people who invented numerological techniques knew that you could manipulate numbers using particular rules and discover new facts about the world -- for example that you could measure the length of strings and use that to determine, which ones would sound good if you plucked them at the same time. Give the large number of successes that mathematics lead to, it only made sense that they would try to encode more things as numbers, including a more general method of translating _all words_ to numbers, using the most straight forward encoding they could think of -- just having the letters stand in for the numbers that the _same letters_ stood for when they were doing math. It didn't develop this way in reality of course, but you could probably draw a fairly straight line of logical developments from numerology to word2vec and LLMs.
kazinator 3y ago
1. Gödel encodes statements about number theory into numbers via Gödel numbering: an arithmetic encoding. 2. Thus we can then talk about properties of expressions as being properties of number. We can make statements which say things like "X has a proof", "Y can be proven false", "Z has no proof", where X, Y and Z are embedded numbers (literals). These literals themselves are expressed symbolically in a way that is susceptible to Gödel numbering. These entire statements are Gödel-encoded and so have numbers; e.g. the Gödel number of "X has a proof" for some given X is some other number W. 3. A Gödel sentence can be constructed which says that "G cannot be proven true", where G is the Gödel number of that sentence itself: in effect, the sentence says "I cannot be proven". The sentence contains a literal G, or perhaps some expression which calculates G. When the Gödel number of the sentence is calculated, it turns out to be G. This is a bit like "This sentence is false", but different. It's not a direct contradiction. "I cannot be proven true" can be taken to be true, without contradiction. The sentence says it has no proof and by golly, none can be found. That allows us to regard it as true and add it as a new axiom. That shows that our system wasn't complete; there are truths that can be expressed in its symbols that have to be treated as new postulates.
farhanhubble 3y ago
It's this melding of logic and computation that is fascinating to me.

pyinstallwoes 3y ago

Take this with the most non-serious mode: In the methods used by Gematria and various arts, there is a similar mechanism at play is there not? Transforming words into quasi-hash coordinate space that is computable and thus able to see what is related or equal?

gregfjohnson 3y ago
Fascinating observation, thanks. Relevant to your comment but diverging farther from the original post: Coincidentally, I am taking a beginner's Kabbalah course right now. I wonder if some of the writers of scripture actually used steganography or other word encodings such as the ones you allude to intentionally. One obvious example of word play at the meta level might be Psalm 119.
- pyinstallwoes 3y ago
  Yeah, along those lines I’ve joked that the Bible admits it’s written by a LLM: “in the beginning was the Word and the Word was with God and the Word was God” - that rings like a recursive declaration statement with Genesis and other similar statements.
empath-nirvana 3y ago
The people who invented numerological techniques knew that you could manipulate numbers using particular rules and discover new facts about the world -- for example that you could measure the length of strings and use that to determine, which ones would sound good if you plucked them at the same time. Give the large number of successes that mathematics lead to, it only made sense that they would try to encode more things as numbers, including a more general method of translating _all words_ to numbers, using the most straight forward encoding they could think of -- just having the letters stand in for the numbers that the _same letters_ stood for when they were doing math. It didn't develop this way in reality of course, but you could probably draw a fairly straight line of logical developments from numerology to word2vec and LLMs.

kazinator 3y ago

1. Gödel encodes statements about number theory into numbers via Gödel numbering: an arithmetic encoding. 2. Thus we can then talk about properties of expressions as being properties of number. We can make statements which say things like "X has a proof", "Y can be proven false", "Z has no proof", where X, Y and Z are embedded numbers (literals). These literals themselves are expressed symbolically in a way that is susceptible to Gödel numbering. These entire statements are Gödel-encoded and so have numbers; e.g. the Gödel number of "X has a proof" for some given X is some other number W. 3. A Gödel sentence can be constructed which says that "G cannot be proven true", where G is the Gödel number of that sentence itself: in effect, the sentence says "I cannot be proven". The sentence contains a literal G, or perhaps some expression which calculates G. When the Gödel number of the sentence is calculated, it turns out to be G. This is a bit like "This sentence is false", but different. It's not a direct contradiction. "I cannot be proven true" can be taken to be true, without contradiction. The sentence says it has no proof and by golly, none can be found. That allows us to regard it as true and add it as a new axiom. That shows that our system wasn't complete; there are truths that can be expressed in its symbols that have to be treated as new postulates.

farhanhubble 3y ago

It's this melding of logic and computation that is fascinating to me.