3 ms·
I performed several thousand batch LLM queries a few months back and asked for such scores. My impression was, while the resulting scores weren't entirely rand
by michaelt 2y ago
I performed several thousand batch LLM queries a few months back and asked for such scores.
My impression was, while the resulting scores weren't entirely random, they were pretty much random when the input was difficult.
(I was mapping supermarket product descriptions to packaging materials - which is sometimes very easy, as sometimes it's obvious or the description includes explicit recycling information, and other times almost impossible, as for products like olive oil, glass and plastic bottles are equally plausible)
- herbst 2y agoI guess that's way more complex because there is no one right answer. (Interesting use case btw) And if you ask gpt for an specific answer to something it can't actually know for sure it gets weird fast. Asking it to guess how much of what it told is based on actual things it 'knows' at least seems like a reasonable question for a LLM with only one right answer.