3 ms·

I don't agree with this rule of thumb at all. Let's say tomorrow someone comes up with a formally verified proof that a major encryption algorithm underpinning

by jbstack 1mo ago

I don't agree with this rule of thumb at all. Let's say tomorrow someone comes up with a formally verified proof that a major encryption algorithm underpinning the security of the internet can be trivially broken, but they can't explain it. You're saying it should be kept under wraps and not published? Tao is essentially saying that the only value in a proof is its ability to be understood, but that's wrong. A proof is also valuable because it establishes a new fact. The fact is useful in itself, even absent an explanation.

jumpman500 1mo ago
I sort of agree, if you can guarantee the AI generated proof isn't a false positive. To be fair though, your example would be easy to explain to someone. You just show how the encryption algorithm can be trivially broken. A program that can break major encryption algorithm would likely be understandable, or at the very least we could show how it can decrypt things.
nixon_why69 1mo ago
Tao is talking about academic publishing, based around learning for it's own sake. If no learning happened, do not publish. From a different angle, what we don't understand can absolutely hurt us and you're right too, but it doesn't contradict Tao's viewpoint. Edit: if you had a PoC cracking the encryption, I think the result is still publishable due to impact and the fact of the empirical result. That's different from some esoteric proof that nobody is even sure it's right.
applecoffeecake 1mo ago
Tao is clearly referring to mathematics journals. Any kind of practically useful result can easily be published in an engineering or applied scientific journal.
feoren 1mo ago
There has already been a formally verified proof of the Collatz Conjecture. The AI agent formally verified it by exploiting previously undiscovered bugs in Lean. The Collatz Conjecture is still unsolved. > Let's say tomorrow someone comes up with a formally verified proof that a major encryption algorithm underpinning the security of the internet can be trivially broken, but they can't explain it. You're saying it should be kept under wraps and not published? Absolutely. It could also be exploiting bugs in the verifier. Even if not -- even if that proof were correct and entirely written by humans, care should still be taken in how such knowledge is published. I'd want to give trusted parties a chance to try to fix the issue before letting it be known by black-hats, for instance.

by jbstack 1mo ago

jumpman500 1mo ago

I sort of agree, if you can guarantee the AI generated proof isn't a false positive. To be fair though, your example would be easy to explain to someone. You just show how the encryption algorithm can be trivially broken. A program that can break major encryption algorithm would likely be understandable, or at the very least we could show how it can decrypt things.

nixon_why69 1mo ago

Tao is talking about academic publishing, based around learning for it's own sake. If no learning happened, do not publish. From a different angle, what we don't understand can absolutely hurt us and you're right too, but it doesn't contradict Tao's viewpoint. Edit: if you had a PoC cracking the encryption, I think the result is still publishable due to impact and the fact of the empirical result. That's different from some esoteric proof that nobody is even sure it's right.

applecoffeecake 1mo ago

Tao is clearly referring to mathematics journals. Any kind of practically useful result can easily be published in an engineering or applied scientific journal.

feoren 1mo ago

There has already been a formally verified proof of the Collatz Conjecture. The AI agent formally verified it by exploiting previously undiscovered bugs in Lean. The Collatz Conjecture is still unsolved. > Let's say tomorrow someone comes up with a formally verified proof that a major encryption algorithm underpinning the security of the internet can be trivially broken, but they can't explain it. You're saying it should be kept under wraps and not published? Absolutely. It could also be exploiting bugs in the verifier. Even if not -- even if that proof were correct and entirely written by humans, care should still be taken in how such knowledge is published. I'd want to give trusted parties a chance to try to fix the issue before letting it be known by black-hats, for instance.