3 ms·
You know that it's not possible to do worse than a coin flip, right? If you're getting it 100% wrong, I'll just do the opposite of what you say, and have a 100%
by stavros 10mo ago
You know that it's not possible to do worse than a coin flip, right? If you're getting it 100% wrong, I'll just do the opposite of what you say, and have a 100% correct predictor.
- waldrews 10mo agoThe threshold isn't 50% because the distribution of human and AI written cases isn't naturally 50-50. So a coin flip will underperform always guessing the more frequent class. Where it gets interesting is if the base is unknown or variable over time or between application domains. Like, since AI written text is being generated faster than the human kind, soon guessing AI every time will be 99% accurate. That doesn't mean such a detector is useful.
- stavros 10mo agoWhen we say "coin flip" in these situations we mean "chance", ie the prior distribution. Otherwise a predictor of the winning lottery numbers that's "no better than a coin flip" would mean it wins the jackpot half the time.
- waldrews 10mo agoYup! My point is that the 'coin flip baseline' model that's as good as chance isn't actually trivial to create, for an unbalanced and time varying underlying distribution.
- thot_experiment 10mo agoOnly if you have that data available to you, the brain to analyze it and the freedom to chose.