3 ms·
A human researcher that is basically right 40%-95% of the time would probably an Einstein level genius. Just assume that the LLM is wrong and test their assump
by portaouflop 2y ago
A human researcher that is basically right 40%-95% of the time would probably an Einstein level genius.
Just assume that the LLM is wrong and test their assumptions - math is one of the few disciplines where you can do that easily
- deleted 2y ago[deleted]
- rednerrus 2y agoIt's pretty easy to test when it makes coding mistakes as well. It's also really good at "Hey that didn't work, here's my error message."
- amanda99 2y agoI think you are imagining a different class of "questions". To clarify, I was doing research on applied math. My field is not analysis, but I needed to prove some bounds on certain messed up expressions (involving special functions, etc), and analyze an ODE that's not analytically solvable. I used the COT model a fair bit. I would ask ChatGPT for hints/ideas/direction in proving various bounds, asking it for theorems or similar results in literature. This is exactly the kind of thing where a researcher would go "yeah this looks like X" or "I think I saw something like this in (book/article name)", or just know a method; or alternatively say they have no clue. ChatGPT most often will confidently give me a "solution", being right 10% of the time (when there's a pretty standard way to do it that I didn't see/know). On the whole it was quite useful.