4 ms·
The paper mentions that on several occasions the LLM will provide a correct answer but will either take big jumps without justifying them or will take illogical
by jtefera 2y ago
The paper mentions that on several occasions the LLM will provide a correct answer but will either take big jumps without justifying them or will take illogical steps but end up with the right solution at the end. Did you check for that?
- e1g 2y agoNo, I don't know enough math to test the logic, only the check questions against their expected answers in https://anonymous.4open.science/r/putnam-axiom-B57C/data/Putnam-AXIOM_Variations_v3/test.json https://anonymous.4open.science/r/putnam-axiom-B57C/data/Put...
- zeroonetwothree 2y agoPutnam problems need to actually be graded, often the answer itself is trivial.