3 ms·
26% of true positive and 9% of false positive is just terrible. I don't see how this can be usable.
by rafaelero 4y ago
26% of true positive and 9% of false positive is just terrible. I don't see how this can be usable.
- yboris 4y agoQuote: > In our evaluations on a “challenge set” of English texts I wonder if they mean "challenge" in the sense that these are some of the hardest-to-discern passages. Meaning that with average human writing / average type of text, the % is better. I'm unsure.
- cuteboy19 4y agoYou are correct. These are handwritten and curated challenges. There are certain terms in ML having very specific meanings which might be missed by casual readers.
- robomc 4y agoit can't be used usefully for anything. the only time it's better than flipping a coin is when there's known to be a majority of human texts in your corpus but even under those conditions it will fail to flag the majority of AI texts. In a set of 100 texts, with 20 being AI, the most likely outcome would be 5 AI texts correctly flagged, along with 7 falsely accused human texts. For like, 22 incorrect answers. For 100 texts where 90 are AI, it would be better to just flip a coin. A coin flip would give you around half correct, and this system would apparently give you around 68 wrong answers (three quarters of the 90 AI ones wrong, then one of the human ones wrong).