3 ms·
Not my article, but this is quite convincing of the brittle nature of these tools. https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-is-broken-but htt
by noah_buddy 2mo ago
Not my article, but this is quite convincing of the brittle nature of these tools.
https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-is-broken-but https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...
- no_multitudes 2mo agoI agree Pangram's UI is awful and often misleading. Their underlying classifier model is pretty accurate in my experience, though, at least in the sense that it has very few natural false positives. If you disagree, please send me some long-form verifiable false positives that were not explicitly written to trick Pangram. I love to learn.
- noah_buddy 2mo agoI just sent you a long article which you could not have read in the time it took you to reply. I would suggest you start there and read that article, which outlines several cases where the author was able to create contrived false positives and negatives.
- no_multitudes 2mo agoI read this article about a week ago. Did you read my reply? I am interested in natural false positives (ones not written to fool the classifier.)
- Matticus_Rex 2mo agoContrived false positives and negatives could (and should!) always be possible. That doesn't tell us anything about the natural false positive and negative rates, and a lot of people who have actively tried to use voice instruction to get models to consistently fool the detector without iterating against it directly have failed.