3 ms·

Without going into the technical efficacy of such schemes ( I am a skeptic), the proposed solution requires the entity generating the media to use it. Isn’t tha

Without going into the technical efficacy of such schemes ( I am a skeptic), the proposed solution requires the entity generating the media to use it. Isn’t that a flaw? Why would an attacker use it willingly? If they did not want to push an AI generated content as a real one, they would have willingly made that distinction themselves. The point is, there is no good solution here unless there is regulation, but these attempts at solutions are useful in the long run

currymj 2y ago
i also doubt practicality. but if the technology worked "well enough" and the major consumer-facing AI companies did it, it would help a lot with certain problems. if you're technically sophisticated you can easily evade it of course. but 1) most people aren't technically sophisticated, 2) one can ask "why are you going through all this effort to remove the watermark if you aren't trying to deceive anyone"?
colmmacc 2y ago
GenAI producers have an incentive to watermark ... it helps them avoid consuming generated output for their own training processes. Most "attackers" aren't going to be sophisticated enough to use a modified non-watermarking tool, and those are likely to fall behind in capability over time anyway. So there's a decent chance here that things could align for watermarking without needing regulation. It probably hinges on whether the stenography can be good enough to avoid being trivially removed or undone.
- jfarina 2y ago
  I disagree that attackers aren't sophisticated enough to use modified tools. There are entire work campuses dedicated to committing fraud. There's also state sponsored subterfuge. There's no reason to think that bad actors are intrinsically unsophisticated.
- amelius 2y ago
  > GenAI producers have an incentive to watermark Yeah in the long run you might be right, but there will be lots of people looking for a quick opportunity. E.g. a content farmer trying to SEO. If Google punishes websites for serving AI generated content, you know where this will end.
- atrus 2y ago
  There's nothing wrong with consuming generated output for training. Blindly accepting trash input is the issue.
- _factor 2y ago
  Did you mean to stay steganography perchance?
ben_w 2y ago
> Why would an attacker use it willingly? Lots of people are very lazy. Citation: all the times we already spot obvious AI-generated comment, which in the early days included the models literally saying "As an AI language model" and this ending up in Amazon reviews etc.
- rvnx 2y ago
  Example: https://twitter.com/KingSafdar441/status/1776041379040309612 https://twitter.com/KingSafdar441/status/1776041379040309612
jazzyjackson 2y ago
Putting on my suspension of disbelief goggles, I've long desired an internet with more cryptographic provenance, just some way so when people share a tweet on some other platform, I know that string was signed by that author, or some photo published by AP is signed by AP. It's just a real "we have the technology" situation. Keybase.io was moving us in this direction before they got acquihired by Zoom. Anyway. In this future, my camera would have its jpgs signed with Nikon or Sony's certificate [0][1], art created by OpenAI would be signed up open AI, and images without any signature would be suspect by default. [0] https://www.dpreview.com/news/6352280282/sony-content-authenticity-system-not-just-for-pro-cameras-anymore https://www.dpreview.com/news/6352280282/sony-content-authen... [1] https://en.wikipedia.org/wiki/Content_Authenticity_Initiative https://en.wikipedia.org/wiki/Content_Authenticity_Initiativ...
exac 2y ago
Regulation makes this worse. Imagine if every gen-AI media was required to have some signature. The bad actors won't use this signature. Or worse, they will add the signature to real media to cast doubt. Or both at the same time. Also, how much gen AI is enough to require a watermark? If I generativley fill the part of my photo that I blurred with my thumb, should the entire photo be watermarked? Just the part that I filled in? What if I use a gen AI tool, but only apply a transformation that does not require gen AI (like adding a label on an image)? Cryptographic watermarks for Gen AI or for non-Gen AI media is dangerous from a security standpoint.

currymj 2y ago

i also doubt practicality. but if the technology worked "well enough" and the major consumer-facing AI companies did it, it would help a lot with certain problems. if you're technically sophisticated you can easily evade it of course. but 1) most people aren't technically sophisticated, 2) one can ask "why are you going through all this effort to remove the watermark if you aren't trying to deceive anyone"?

colmmacc 2y ago

GenAI producers have an incentive to watermark ... it helps them avoid consuming generated output for their own training processes. Most "attackers" aren't going to be sophisticated enough to use a modified non-watermarking tool, and those are likely to fall behind in capability over time anyway. So there's a decent chance here that things could align for watermarking without needing regulation. It probably hinges on whether the stenography can be good enough to avoid being trivially removed or undone.

jfarina 2y ago
I disagree that attackers aren't sophisticated enough to use modified tools. There are entire work campuses dedicated to committing fraud. There's also state sponsored subterfuge. There's no reason to think that bad actors are intrinsically unsophisticated.
amelius 2y ago
> GenAI producers have an incentive to watermark Yeah in the long run you might be right, but there will be lots of people looking for a quick opportunity. E.g. a content farmer trying to SEO. If Google punishes websites for serving AI generated content, you know where this will end.
atrus 2y ago
There's nothing wrong with consuming generated output for training. Blindly accepting trash input is the issue.
_factor 2y ago
Did you mean to stay steganography perchance?

ben_w 2y ago

> Why would an attacker use it willingly? Lots of people are very lazy. Citation: all the times we already spot obvious AI-generated comment, which in the early days included the models literally saying "As an AI language model" and this ending up in Amazon reviews etc.

rvnx 2y ago
Example: https://twitter.com/KingSafdar441/status/1776041379040309612 https://twitter.com/KingSafdar441/status/1776041379040309612

jazzyjackson 2y ago

Putting on my suspension of disbelief goggles, I've long desired an internet with more cryptographic provenance, just some way so when people share a tweet on some other platform, I know that string was signed by that author, or some photo published by AP is signed by AP. It's just a real "we have the technology" situation. Keybase.io was moving us in this direction before they got acquihired by Zoom. Anyway. In this future, my camera would have its jpgs signed with Nikon or Sony's certificate [0][1], art created by OpenAI would be signed up open AI, and images without any signature would be suspect by default. [0] https://www.dpreview.com/news/6352280282/sony-content-authenticity-system-not-just-for-pro-cameras-anymore https://www.dpreview.com/news/6352280282/sony-content-authen... [1] https://en.wikipedia.org/wiki/Content_Authenticity_Initiative https://en.wikipedia.org/wiki/Content_Authenticity_Initiativ...

exac 2y ago

Regulation makes this worse. Imagine if every gen-AI media was required to have some signature. The bad actors won't use this signature. Or worse, they will add the signature to real media to cast doubt. Or both at the same time. Also, how much gen AI is enough to require a watermark? If I generativley fill the part of my photo that I blurred with my thumb, should the entire photo be watermarked? Just the part that I filled in? What if I use a gen AI tool, but only apply a transformation that does not require gen AI (like adding a label on an image)? Cryptographic watermarks for Gen AI or for non-Gen AI media is dangerous from a security standpoint.