3 ms·
It only spots labellers that are not in the middle of the curve of those who you employ. My point was, if factors like salary affect labelling quality, how do
by layer8 4y ago
It only spots labellers that are not in the middle of the curve of those who you employ. My point was, if factors like salary affect labelling quality, how do you steer that? You don’t know whether if you’d pay double, you’d get much better fact-checkers, for example.
- kelseyfrog 4y agoInterater and intrarater reliability are two general approaches. You could even regress reliability with salary and make a case that reliability would improve by a certain amount(within a prediction interval), but that's a business decision to act on, not a statistics problem.