"A team of 17 annotators completed a pilot annotation exercise of 50 annotations per task. These pilot annotations were checked and verified by the authors to p

"A team of 17 annotators completed a pilot annotation exercise of 50 annotations per task. These pilot annotations were checked and verified by the authors to pinpoint common sources of error and disagreement." All good but I wonder how this is supposed to scale? As they also do acknowledge: "Our work is motivated by the need for efficient LLM hallucination evaluation. Though computationally efficient, one of the key limitations of probing is in the need for labeled in-domain data for probe training."