3 ms·
The title is like saying "The profit increases by 0%", which is grammatically correct and logically sound, but that exactly means the profit doesn't increase at
by beltsazar 4y ago
The title is like saying "The profit increases by 0%", which is grammatically correct and logically sound, but that exactly means the profit doesn't increase at all.
When the task is choosing between two choices (in this case: AI/Human), the worst you can do in average is not 0% correct, but 50%, which is a coin flip. If a model—whether it's an ML one or is inside human's mind—achieves 40% accuracy in a binary prediction, it can increases the accuracy to 60% by just flipping the answers.
The more interesting numbers are precision and recall, or even better, a confusion matrix. It might turn out that the false AI score and the false human score (in the sense of false positive/negative) differ significantly. That would be a more interesting report.
- playingalong 4y agoWait. If your job is to detect AI vs. human and you happen to be always wrong, then your score is 0%. Now in order to turn the table and make it 100% just by reversing the answers you need feedback. Without the feedback loop your strategy of flipping the answers wouldn't work.
- beltsazar 4y ago> happen to be always wrong Which is very very unlikely. When you don't know the answer, you'll guess, which is like flipping a coin. Suppose you have zero knowledge and always guess the answer. Can you get unlucky that you get more wrong answers than correct answers? Sure. But the more questions you get, the closer your accuracy is to 50%. Another way to look at it is to realize that a binary prediction has the highest entropy (uncertainty) when it splits fifty-fifty.