> Follow-up analysis of the first experimental results revealed a marked positional bias with LLMs tending to prefer the candidate appearing first in the prompt

> Follow-up analysis of the first experimental results revealed a marked positional bias with LLMs tending to prefer the candidate appearing first in the prompt: 63.5% selection of first candidate vs 36.5% selections of second candidate To my eyes this ordering bias is the most glaring limitation of LLMs not only within hiring but also applications such as RAG or classification: these applications often implicitly assume that the LLMs is weighting the entire context evenly: the answers are not obviously wrong, but they are not correct because they do not take the full context into account. The lost in the middle problem for facts retrieval is a good correlative metric, but the ability to find a fact in an arbitrary location is not the to same as the ability to evenly weight the full context