3 ms·
All that's happening is it finds 3 most commonly in the training set. When you push it, it responds with the next most common answer.
by brandall10 2y ago
All that's happening is it finds 3 most commonly in the training set. When you push it, it responds with the next most common answer.
- paulcole 2y agoBut then why does it stick to its guns on other questions but not this one?
- brandall10 2y agoI haven't played with this model, but rarely do I find working w/ Claude or GPT-4 for that to be the case. If you say it's incorrect, it will give you another answer instead of insisting on correctness.
- paulcole 2y agoWait what? You haven’t used 4o and you confidently described how it works?
- brandall10 2y agoIt's how LLMs work in general. If you find a case where forceful pushback is sticky, it's either because the primary answer is overwhelmingly present in the training set compared to the next best option or because there are conversations in the training that followed similar stickiness, esp. if the structure of the pushback itself is similar to what is found in those conversations.
- paulcole 2y agoRight... except you said: > If you say it's incorrect, it will give you another answer instead of insisting on correctness. > When you push it, it responds with the next most common answer. Which clearly isn't as black and white as you made it seem.
- brandall10 2y agoI'll put it another way - behavior like this is extremely rare in my experience. I'm just trying to explain if one encounters it why it's likely happening.