3 ms·
> Positional preferences, order effects, prompt sensitivity undermine AI judgments If you can read between the lines, that says that there's no actual "judgeme
by SrslyJosh 1y ago
> Positional preferences, order effects, prompt sensitivity undermine AI judgments
If you can read between the lines, that says that there's no actual "judgement" going on. If there was a strong logical underpinning to the output, minor differences in input like the phrasing (but not factual content) of a prompt wouldn't make the quality of the output unpredictable.
- Wilsoniumite 1y agoYes and no. People also exhibit these biases, but because degree matters, and because we have no other choice, we still trust them most of the time. That's to say; bias isn't always completely invalidating. I wrote a slightly satirical piece "People are just as bad as my LLMs" here: https://wilsoniumite.com/2025/03/10/people-are-just-as-bad-as-my-llms/ https://wilsoniumite.com/2025/03/10/people-are-just-as-bad-a...
- tiahura 1y agoWord plinko
- ACCount36 1y agoYou could say the same about human "judgement" then. Humans display biases very similar to that of LLMs. This is not a coincidence. LLMs are trained on human-generated data. They attempt to replicate human reasoning - bias and all. There are decisions where "strong logical underpinning" is strong enough to completely drown out the bias and the noise. And then there are decisions that aren't nearly as clear-cut - allowing the bias and the noise to creep into the outcomes. This is true for human and LLM "judgement" both.