3 ms·
the fact that you have to encourage these models and tell them that they can solve these problems and warm up on easier problems seems to indicate that there's
by htrp 19d ago
the fact that you have to encourage these models and tell them that they can solve these problems and warm up on easier problems seems to indicate that there's something to AI pairing above and beyond prompt
- bananaflag 19d agoYeah it s because they have a strong prior on unsolved problems being unsolvable. Once the idea of AI routinely solving conjectures enters the training data this encouragement will disappear like 2023-era prompt engineering did.
- arscan 19d agoI would think this type of behavior (ugh, or dare I say default mindset) by consumer-facing LLMs will always be desirable for ‘hard’ problems (things previously unsolved) because it’s a bit like having saftey mechanisms in place to prevent hallucinations for users incapable of verifying correctness of the output. You’ve got to do a little work to prove you understand that it’s hard but it’s still something the LLM might be able to accomplish.
- rowanG077 19d agoI think this is also a healthy mindset for a person to have in many cases. If my boss asks me "go solve P = NP", as extreme example, I would also give some pushback.
- skybrian 19d agoIt will likely burn a lot of tokens, take a long time, and might not work, so hopefully they'll still ask if you really want to spend the money on the attempt.
- empath75 19d agoI spent about a month walking through proving something with Claude a few months ago and it _constantly_ told me that it was impossible and I should stop working on it, right up until it proved it.
- asolove 19d agoThey are relying on training data of humans talking about how hard these problems are. Same way an un-reminded Claude gives estimates for work that are as if a human is doing it by hand, but then will drop them by 20x if you remind them it’s going to do the work.
- dist-epoch 19d agoOne way they improved the hallucination problem was basically training the models to refuse to do or say something if they are not very sure they can do it. As a side effect, they refuse to work on problems they know are extremely hard.