4 ms·
Identifying Stable Diffusion XL 1.0 images from VAE artifacts (2023)
- blt 3y ago(2023)
- dang 3y agoAdded. Thanks!
- GaggiX 3y ago8 months ago StabilityAI changed the default VAE from 1.0 to 0.9, so no one realistically uses VAE 1.0.
- kken 3y agoGenerally, the VAE is mapping from a small latent space to a large image space. This means that there must be a large number of images for which no reverse mapping exists. It should be possible to identify images that have not been generate by the VAE since they are not part of the set images that the VAE can generate. The other way round is a bit more difficult as there may be images that can be mapped to the latent space and back without loss but have been generated in another way -> there may be false positives.
- chacham15 3y agoThis logic has a key flaw: just the fact that the size of the space is different doesnt mean that every representable thing in the larger space is a thing we care about. E.g. a person with three hands may not have a representation in the smaller space, but we would never care about that. The actual question is: what is the difference in the amount of information encoded in a large image vs the small latent space and compare that to the difference in information between a large image and a small image. If those two differences are close enough together, being able to determine a legitimate difference between SD generated vs not becomes near impossible.
- TrueDuality 3y agoVery interesting! Well broken down and explained!
- tsycho 3y agoHas anyone tried training a neural net to distinguish between photographed vs AI generated images? Of course, you will need to remove exifs or other metadata, but this sounds like the kind of domain that NNs are good at.
- moofight 3y agoyes, and it's quite challenging for instance => https://sightengine.com/detect-ai-generated-images https://sightengine.com/detect-ai-generated-images
- HPsquared 3y agoIsn't that basically how the image generation models themselves work? By refining the image until it can't distinguish it from a "real image".
- jncfhnb 3y agoThat was trendy for a while but is no longer the primary method
- ok123456 3y agoYes. Trying to sell an "AI detector" is a fool's errand since this is how adversarial networks (the hot model from two hype cycles ago) are trained. The only use of the "AI detector " is to tune the model so that the detector is uniformly random.
- mindcandy 3y agoThis is true for explicitly adversarial actors. I can imagine a serious-business misinformation group investing the money required to train up a generator designed to defeat automatic detection. For everyone else though, the people making common tools everyone uses explicitly want their images to be easy to automatically ID as fake. And, the users largely prefer it that way too.
- 3y ago
- 29athrowaway 3y agoIf you have a scoring system that tells you how "AI-like" a picture is... congrats, you built a loss function that can be used to train AI.
- Dwedit 3y agoTelling how "AI-like" an image is is done by seeing how well it completes a round-trip through the VAE. It will either match a known VAE well, or be an image that's altered enough (jpeg noise, photoshopping in post, etc) to not match the output of a VAE. And meanwhile the "loss system" for training AI idea is still hobbled by how the image generating AI systems are all using VAEs. At best, you've swapped one VAE for another. Just another new VAE to add to the list of VAEs to check.
- borgchick 3y agoAnd the next round of spy vs spy begin... AI generated content vs detecting it... just like the age old email spam problem.
- Zetobal 3y agoIt falls short because you can just transpose the latents and use whatever vae you want.