3 ms·

Ask it to formalize it in Lean.

dbdr 7mo ago
That's great if it works. But it's way harder to produce a formal proof. So my expectation is that this will fail for most difficult problems, even when the non-formal proof is correct.
utopiah 7mo ago
If they aren't "smart enough" to know if it work they most likely are also unable to verify if the Lean formalization is indeed the one that matches the problem they were trying to solve.
- timjver 7mo ago
  Verifying that every step in a (potentially long) proof is sound can of course be much, much harder than verifying that a definition is correct. That's kind of the whole point.
  - LeCompteSftware 7mo ago
    That's not what the parent comment meant. They meant checking the Lean-language definitions actually match the mathematical English ones, and that the Lean theorems match the ones in the paper. If that's true then you don't actually need to check the proofs. But you absolutely need to check the definitions, and you can't really do that without sufficient mathematical maturity.
    - smallnamespace 7mo ago
      Yes, and the child comment’s point is that formalizing the problem is likely easier than having the LLM verify that each step of a long deduction is correct, which is why Lean might be helpful.
      - LeCompteSftware 7mo ago
        But both of you are ignoring the parent comment! Actually you're ignoring the context of the thread. Originally someone said "I wish I was math smart to know if [this vibe-mathematics proof] worked or not." They did NOT say "I'd like to check but I am too lazy." Suggesting "ask it to formalize it in Lean" is useless if you're not mathematically mature enough to understand the proof, since that means you're not mathematically mature enough to understand how to formalize the problem. Then "likely easier" is a moot point. A Lean program you're not knowledgeable enough to sanity-check is precisely as useless as a math proof you're not knowledgeable enough to read.
        utopiah 7mo ago
        thanks
        smallnamespace 7mo ago
        It’s not useless, because you can, for example, ask multiple frontier models to do the formalization and see if they agree. And if they have surface-level differences in formalization, you can also ask them whether apparently-different definitions are equivalent. This isn’t perfect of course - perhaps every single model is wrong. But you are too quick to declare that something isn’t useful for arriving at an answer. Reducing the surface area of what needs to be checked is good regardless.
DonHopkins 7mo ago
Formalize this in the form of a Iranian Lego Trump Dis Rap video.

dbdr 7mo ago

That's great if it works. But it's way harder to produce a formal proof. So my expectation is that this will fail for most difficult problems, even when the non-formal proof is correct.

utopiah 7mo ago

If they aren't "smart enough" to know if it work they most likely are also unable to verify if the Lean formalization is indeed the one that matches the problem they were trying to solve.

timjver 7mo ago
Verifying that every step in a (potentially long) proof is sound can of course be much, much harder than verifying that a definition is correct. That's kind of the whole point.
- LeCompteSftware 7mo ago
  That's not what the parent comment meant. They meant checking the Lean-language definitions actually match the mathematical English ones, and that the Lean theorems match the ones in the paper. If that's true then you don't actually need to check the proofs. But you absolutely need to check the definitions, and you can't really do that without sufficient mathematical maturity.
  - smallnamespace 7mo ago
    Yes, and the child comment’s point is that formalizing the problem is likely easier than having the LLM verify that each step of a long deduction is correct, which is why Lean might be helpful.
    - LeCompteSftware 7mo ago
      But both of you are ignoring the parent comment! Actually you're ignoring the context of the thread. Originally someone said "I wish I was math smart to know if [this vibe-mathematics proof] worked or not." They did NOT say "I'd like to check but I am too lazy." Suggesting "ask it to formalize it in Lean" is useless if you're not mathematically mature enough to understand the proof, since that means you're not mathematically mature enough to understand how to formalize the problem. Then "likely easier" is a moot point. A Lean program you're not knowledgeable enough to sanity-check is precisely as useless as a math proof you're not knowledgeable enough to read.
      - utopiah 7mo ago
        thanks
      - smallnamespace 7mo ago
        It’s not useless, because you can, for example, ask multiple frontier models to do the formalization and see if they agree. And if they have surface-level differences in formalization, you can also ask them whether apparently-different definitions are equivalent. This isn’t perfect of course - perhaps every single model is wrong. But you are too quick to declare that something isn’t useful for arriving at an answer. Reducing the surface area of what needs to be checked is good regardless.

DonHopkins 7mo ago

Formalize this in the form of a Iranian Lego Trump Dis Rap video.