3 ms·
Ask GPT4o to return a percentual score after each answer how likely something is 100% true. Did that for old regional stories and the higher the score the easi
by herbst 2y ago
Ask GPT4o to return a percentual score after each answer how likely something is 100% true.
Did that for old regional stories and the higher the score the easier I could verify the story or parts of it. It seems to know, at least in some cases
- michaelt 2y agoI performed several thousand batch LLM queries a few months back and asked for such scores. My impression was, while the resulting scores weren't entirely random, they were pretty much random when the input was difficult. (I was mapping supermarket product descriptions to packaging materials - which is sometimes very easy, as sometimes it's obvious or the description includes explicit recycling information, and other times almost impossible, as for products like olive oil, glass and plastic bottles are equally plausible)
- herbst 2y agoI guess that's way more complex because there is no one right answer. (Interesting use case btw) And if you ask gpt for an specific answer to something it can't actually know for sure it gets weird fast. Asking it to guess how much of what it told is based on actual things it 'knows' at least seems like a reasonable question for a LLM with only one right answer.
- user-the-name 2y ago[dead]