3 ms·

Don't most AI either incorporate or surpass GAN architecture, making good software detector impossible to build? If I understand right, GAN is a co-trained AI

Don't most AI either incorporate or surpass GAN architecture, making good software detector impossible to build? If I understand right, GAN is a co-trained AI similar in size to main AI that discriminate and identify generated AI data apart from sample human data. When the main AI has gotten so good that GAN becomes a dice roll, training is considered finished. Human brains are completely different implementation and nothing is stopping us from instantly spotting AI, at least for now, but computers shouldn't be able to. Isn't that the case?

blah2244 2y ago
I've heard of a startup that claims to be able to achieve a near-0% false positive rate: https://www.pangram.com/our-model/how-it-works https://www.pangram.com/our-model/how-it-works They appear to basically RLHF a model on a bunch of examples of human/AI output on the same prompt. Not sure how well it works, but I'm guessing Mozilla is doing something similar here.
- saretup 2y ago
  Anyone can get a 0% false positive by always inferring negative. What you wanna look at is the precision-recall curve. https://en.m.wikipedia.org/wiki/Precision_and_recall https://en.m.wikipedia.org/wiki/Precision_and_recall
godelski 2y ago
> Don't most AI either incorporate or surpass GAN architecture No. Most generators don't bother because it is time consuming. Maybe they will if there is a popular detector like this. But it sure increases the cost of training and do people really care? Probably not unless you're explicitly trying to create subversive content. It's also important to remember that tuning a model can also have adverse effects. Sure, you get better in one way, but that can often mean you get worse in another. There's no free lunch. To "Um AcKtUaLlY" you, GAN is actually a training technique, not an architecture, though we all know what you mean. Maybe worth noting, that our GAN based networks tend to result in much smaller and faster generative models which can thus produce higher resolutions with much lower resources. The best face generators are still GAN based btw. But you still need big architectures to match diffusion models and you tend to lose your advantages (not all, but it gets nuanced). > making good software detector impossible to build? It is a cat and mouse game. I'm not sure there is anything wrong with this, but detector systems tend not to be continually updated, aimed at specific models, and I think their communication is very unclear. You should never say anything is 100% generated, it should be "suspect." So "low suspect" to "high suspect" but never "not AI generated" or "AI generated." The communication is problematic and leads to kids getting flagged for plagiarism when they just wrote something generic. Though tbh, with a lot of the LLM based detectors, I am really confused by the methodologies. Like why is there this big focus around perplexity? LLMs are trained to be entropy minimizers, so it really doesn't make sense to me that detectors are often looking for... entropy (since perplexity is scaled entropy).

blah2244 2y ago

I've heard of a startup that claims to be able to achieve a near-0% false positive rate: https://www.pangram.com/our-model/how-it-works https://www.pangram.com/our-model/how-it-works They appear to basically RLHF a model on a bunch of examples of human/AI output on the same prompt. Not sure how well it works, but I'm guessing Mozilla is doing something similar here.

saretup 2y ago
Anyone can get a 0% false positive by always inferring negative. What you wanna look at is the precision-recall curve. https://en.m.wikipedia.org/wiki/Precision_and_recall https://en.m.wikipedia.org/wiki/Precision_and_recall

godelski 2y ago

> Don't most AI either incorporate or surpass GAN architecture No. Most generators don't bother because it is time consuming. Maybe they will if there is a popular detector like this. But it sure increases the cost of training and do people really care? Probably not unless you're explicitly trying to create subversive content. It's also important to remember that tuning a model can also have adverse effects. Sure, you get better in one way, but that can often mean you get worse in another. There's no free lunch. To "Um AcKtUaLlY" you, GAN is actually a training technique, not an architecture, though we all know what you mean. Maybe worth noting, that our GAN based networks tend to result in much smaller and faster generative models which can thus produce higher resolutions with much lower resources. The best face generators are still GAN based btw. But you still need big architectures to match diffusion models and you tend to lose your advantages (not all, but it gets nuanced). > making good software detector impossible to build? It is a cat and mouse game. I'm not sure there is anything wrong with this, but detector systems tend not to be continually updated, aimed at specific models, and I think their communication is very unclear. You should never say anything is 100% generated, it should be "suspect." So "low suspect" to "high suspect" but never "not AI generated" or "AI generated." The communication is problematic and leads to kids getting flagged for plagiarism when they just wrote something generic. Though tbh, with a lot of the LLM based detectors, I am really confused by the methodologies. Like why is there this big focus around perplexity? LLMs are trained to be entropy minimizers, so it really doesn't make sense to me that detectors are often looking for... entropy (since perplexity is scaled entropy).