3 ms·

I would imagine that models could be trained on specific image generators to detect them. This is honestly the obvious solution; AI is actually good at finding

I would imagine that models could be trained on specific image generators to detect them. This is honestly the obvious solution; AI is actually good at finding very subtle patterns.

hiddencost 3y ago
Adversarial setting. People will train models on adversarial detectors to hide from them. The comments in this article have deeply studied the method you're suggesting.
- ang_cire 3y ago
  Which is pointless in targeted models like I'm talking about. If I train a model that is *only* trying to detect e.g. stable-diffusion-2-1, training an adversarial model is pointless, because then you're just using a different model anyways. You would be spending a lot of training time when you could just use another already-extant model. State actors, for example, don't need to use public models for their generators anyways, so it's not like non-state actors would have true-positive images to train models on to counter said actors' models.
agumonkey 3y ago
wouldn't this feed a race to generators being even closer to indistinguishable ?
- ang_cire 3y ago
  Potentially, but I don't think in practice, because most image generation is not trying to be photorealistic. Most models are trying to stylize their outputs, either by default (e.g. a Miyazaki-anime-like model), or in response to the prompt. Using AI image generation for misinformation is by far the vast minority of its use. And if the output is stylized, it's going to be much more obvious if it has come from an AI model, because a human is going to have a much harder time reproducing a specific ML model's style on-demand (e.g. if their art teacher asks them to sketch a face in the same style, to prove their homework wasn't faked).
jvanderbot 3y ago
The problem is this isn't like cryptography where you can prove something. You can train a detector on a generator, or at least what you think the generator is, but you can never know for sure. And any "likely fake" assessment can just be refuted by a sock puppet that vouches for the content.
willsmith72 3y ago
This is the main discussion point of the article > Just as machines can be trained to reliably identify cats, or cancerous tumours on medical scans, they can also be trained to differentiate between real images and ai-generated ones. But the conclusion the researchers made was detection at some point wont be reliable and fakes will win
- ang_cire 3y ago
  The tools the article discusses are *general* AI-detection, meant to be fed any image and determine whether it was AI-generated. I'm talking about a detection model that only detects images produced by one generator/source model, so rather than a one-to-many process of having one tool check an image for many potential source generators' artifacts, you'd have many individually-trained detection models, each being run against a single image/text block.

I would imagine that models could be trained on specific image generators to detect them. This is honestly the obvious solution; AI is actually good at finding very subtle patterns.

hiddencost 3y ago

Adversarial setting. People will train models on adversarial detectors to hide from them. The comments in this article have deeply studied the method you're suggesting.

ang_cire 3y ago
Which is pointless in targeted models like I'm talking about. If I train a model that is *only* trying to detect e.g. stable-diffusion-2-1, training an adversarial model is pointless, because then you're just using a different model anyways. You would be spending a lot of training time when you could just use another already-extant model. State actors, for example, don't need to use public models for their generators anyways, so it's not like non-state actors would have true-positive images to train models on to counter said actors' models.

agumonkey 3y ago

wouldn't this feed a race to generators being even closer to indistinguishable ?

ang_cire 3y ago
Potentially, but I don't think in practice, because most image generation is not trying to be photorealistic. Most models are trying to stylize their outputs, either by default (e.g. a Miyazaki-anime-like model), or in response to the prompt. Using AI image generation for misinformation is by far the vast minority of its use. And if the output is stylized, it's going to be much more obvious if it has come from an AI model, because a human is going to have a much harder time reproducing a specific ML model's style on-demand (e.g. if their art teacher asks them to sketch a face in the same style, to prove their homework wasn't faked).

jvanderbot 3y ago

The problem is this isn't like cryptography where you can prove something. You can train a detector on a generator, or at least what you think the generator is, but you can never know for sure. And any "likely fake" assessment can just be refuted by a sock puppet that vouches for the content.

willsmith72 3y ago

This is the main discussion point of the article > Just as machines can be trained to reliably identify cats, or cancerous tumours on medical scans, they can also be trained to differentiate between real images and ai-generated ones. But the conclusion the researchers made was detection at some point wont be reliable and fakes will win

ang_cire 3y ago
The tools the article discusses are *general* AI-detection, meant to be fed any image and determine whether it was AI-generated. I'm talking about a detection model that only detects images produced by one generator/source model, so rather than a one-to-many process of having one tool check an image for many potential source generators' artifacts, you'd have many individually-trained detection models, each being run against a single image/text block.