4 ms·
>The logical calculus is easier to execute than any of the techniques of mathematics itself, yet only in 1934 did Gerhard Gentzen set it out in a natural way. E
by llm_trw 2y ago
>The logical calculus is easier to execute than any of the techniques of mathematics itself, yet only in 1934 did Gerhard Gentzen set it out in a natural way. Even now, mathematics students are expected to learn complicated (epsilon, delta)-proois in analysis with no help in understanding the logical structure of the arguments. Examiners fully deserve the garbage that they get in return.
With an opening like that how can I not read the rest of the book?
This actually touches on some work I've been doing the past week - arbitrary term rewriting to test how reliable a given LLM is when it needs to reason about symbolic manipulation. Everything is dynamically generated to avoid the usual problem of memorising every validation set every llm seems to suffer from.
- memhole 2y agoAnything public you can share yet? I’ve been interested in how much LLMs “understand” symbolism.
- bwfan123 2y agoI am interested in this problem as well. Please share any notes. I am attempting to create parameterized "logic" problems (similar to the zebra puzzle) which cannot be solved by LLMs even when they are trained on it, or even when they "reason" on it. Meanwhile this approach is even simpler, where it is demonstrated that LLMs cannot recognize 3 state DFAs. https://arxiv.org/pdf/2501.02825 https://arxiv.org/pdf/2501.02825
- llm_trw 2y agoYou can email me if you're interested. This was paid work until recently but the start up pivoted to something else and now I'm wondering what to do with code that takes a few thousand dollars to run every time and is of questionable copyright ownership.