4 ms·

>Ensuring that consumers are able to spot A.I.-generated material by implementing watermarks or other means of identifying generated content. This will either

>Ensuring that consumers are able to spot A.I.-generated material by implementing watermarks or other means of identifying generated content. This will either be impossible or something we find out the NSA invented 10 years ago. Would end up being amazing if we could use these to de-enshittify the internet by automatically removing/filtering existing content which triggers

Ethcad 3y ago
We need regulations requiring mandatory watermarks on the outputs of troll bots
dathinab 3y ago
as long as it's about content they generated which weren't modified (at most cropped, scaled) this is quite viable you e. g. can encode a subtle pattern in the generated image which surives compression and isn't really human visible then you make a browser extension to spot that pattern and indicate it to the users "in some way" given that there is a overlap between AI company owners and biggest browser producers and mobile OS vendors this doesn't even need to be an extension but can be build in obviously any bad actor is likely able to remove it or otherwise still trick users
- crazygringo 3y ago
  > encode a subtle pattern in the generated image which surives compression and isn't really human visible This is basically a contradiction in terms. Compression attempts to throw away any and all data that "isn't really human visible," that's how it works. There isn't space for invisible watermarks by design. You can kind of get away with something "at the edge" that survives an initial JPEG encoding, but there's no way it's going to reliably survive e.g. resizing, cropping, and recompressing and still remain invisible. Also, most AI generation content is presumably going to be text, not images. Good luck watermarking text that's a paragraph long. (There are potential tools that can operate on text the size of a news article, but are also trivially defeated by swapping a few prepositions and synonyms.)
  - jjtheblunt 3y ago
    Are you describing dct in particular?
    - ke88y 3y ago
      I would like to know the same thing. In the case of JPEG and similar schemes in particular, an impossibility result that isn't unrealistically narrowly scoped (again assuming non-adversarial user) would be highly surprising.
  - dathinab 3y ago
    > watermarking text that's a paragraph lon unicode has a ton of room for that > There isn't space for invisible watermarks by design. if it is impossible, why does it exist?
  - ke88y 3y ago
    > There isn't space for invisible watermarks by design. Very incorrect. Steghide [1] supports JPEG. JPEG and other lossy image formats are ultimately just fancy file formats; there's nothing preventing you from encoding arbitrary messages in a compressed image. > You can kind of get away with something "at the edge" that survives an initial JPEG encoding, but there's no way it's going to reliably survive e.g. resizing, cropping, and recompressing and still remain invisible. I am pretty sure that I can design steganagraphy algorithm that disperses a small message across a JPEG in a way that is: 1. invariant to resizing (absolutely certain this is possible), 2. robust to cropping (invariant to cropping up to some limit is definitely possible; eg if you crop 100% of the image then obviously everything goes out the window), 3. robust or even invariant to recompression. This seems a lot harder but I'm pretty sure it's possible. > Also, most AI generation content is presumably going to be text, not images. Good luck watermarking text that's a paragraph long. (There are potential tools that can operate on text the size of a news article, but are also trivially defeated by swapping a few prepositions and synonyms.) Yeah, text seems more difficult. Images are also difficult/impossible if you assume the model user is adversarial and competent, which I'm not sure what you wouldn't assume. For any particular model you can probably do detection with a fair bit of inaccuracy. But I would definitely put detection in the "doomed" category. I also think the threat is real but wildly over-stated relative to the non-AI status quo. We're slightly democratizing Photoshop and copywriting skills, which weren't exactly scarce to begin with. It's not an AI problem, and it's barely a technology problem. It's primarily a political problem. [1] https://github.com/StefanoDeVuono/steghide https://github.com/StefanoDeVuono/steghide
- danShumway 3y ago
  > then you make a browser extension to spot that pattern and indicate it to the users "in some way" If that's an Open standard, and if that browser extension is Open Source, then anyone who wants to avoid that can mess with the final image until the Open Source free standard that everyone is using no longer detects the image. The only way this is viable is with DRM or a closed service; if it's a standard everyone follows, then circumventing it is trivial. The only way it would work is if it's shrouded in secrecy and attackers can't freely use red-team against the tool. These kinds of watermarks work when there's a very limited pool of people checking for the watermarks, they don't tell people who the watermarks are generated, and they don't tell people how to check for the watermarks. But that's not really useful for the current situation -- we don't want to further entrench these companies and we don't want it to be costly to check if an image is AI-generated. I don't think it's viable to do this without significantly curtailing user agency or designing a system that is fully opaque and inaccessible to most people.
  - ke88y 3y ago
    Yeah, this is my general feeling about why this area is doomed. It's why I haven't bothered to write up a patent even though I had some good ideas a few years ago. Maybe that was a mistake since governments and corporate politicians are stupider than I assumed. If you have a central source of authority then the problem is totally trivial and the fact that the images are AI-generated (or not) is a complete red herring. If you don't have a central source of authority then any reasonable adversarial model makes the watermarking problem somewhere between very difficult and impossible. Detection from known models is still possible, at least for images. But that's not really watermarking per se.

Ethcad 3y ago

We need regulations requiring mandatory watermarks on the outputs of troll bots

dathinab 3y ago

as long as it's about content they generated which weren't modified (at most cropped, scaled) this is quite viable you e. g. can encode a subtle pattern in the generated image which surives compression and isn't really human visible then you make a browser extension to spot that pattern and indicate it to the users "in some way" given that there is a overlap between AI company owners and biggest browser producers and mobile OS vendors this doesn't even need to be an extension but can be build in obviously any bad actor is likely able to remove it or otherwise still trick users

crazygringo 3y ago
> encode a subtle pattern in the generated image which surives compression and isn't really human visible This is basically a contradiction in terms. Compression attempts to throw away any and all data that "isn't really human visible," that's how it works. There isn't space for invisible watermarks by design. You can kind of get away with something "at the edge" that survives an initial JPEG encoding, but there's no way it's going to reliably survive e.g. resizing, cropping, and recompressing and still remain invisible. Also, most AI generation content is presumably going to be text, not images. Good luck watermarking text that's a paragraph long. (There are potential tools that can operate on text the size of a news article, but are also trivially defeated by swapping a few prepositions and synonyms.)
- jjtheblunt 3y ago
  Are you describing dct in particular?
  - ke88y 3y ago
    I would like to know the same thing. In the case of JPEG and similar schemes in particular, an impossibility result that isn't unrealistically narrowly scoped (again assuming non-adversarial user) would be highly surprising.
- dathinab 3y ago
  > watermarking text that's a paragraph lon unicode has a ton of room for that > There isn't space for invisible watermarks by design. if it is impossible, why does it exist?
- ke88y 3y ago
  > There isn't space for invisible watermarks by design. Very incorrect. Steghide [1] supports JPEG. JPEG and other lossy image formats are ultimately just fancy file formats; there's nothing preventing you from encoding arbitrary messages in a compressed image. > You can kind of get away with something "at the edge" that survives an initial JPEG encoding, but there's no way it's going to reliably survive e.g. resizing, cropping, and recompressing and still remain invisible. I am pretty sure that I can design steganagraphy algorithm that disperses a small message across a JPEG in a way that is: 1. invariant to resizing (absolutely certain this is possible), 2. robust to cropping (invariant to cropping up to some limit is definitely possible; eg if you crop 100% of the image then obviously everything goes out the window), 3. robust or even invariant to recompression. This seems a lot harder but I'm pretty sure it's possible. > Also, most AI generation content is presumably going to be text, not images. Good luck watermarking text that's a paragraph long. (There are potential tools that can operate on text the size of a news article, but are also trivially defeated by swapping a few prepositions and synonyms.) Yeah, text seems more difficult. Images are also difficult/impossible if you assume the model user is adversarial and competent, which I'm not sure what you wouldn't assume. For any particular model you can probably do detection with a fair bit of inaccuracy. But I would definitely put detection in the "doomed" category. I also think the threat is real but wildly over-stated relative to the non-AI status quo. We're slightly democratizing Photoshop and copywriting skills, which weren't exactly scarce to begin with. It's not an AI problem, and it's barely a technology problem. It's primarily a political problem. [1] https://github.com/StefanoDeVuono/steghide https://github.com/StefanoDeVuono/steghide
danShumway 3y ago
> then you make a browser extension to spot that pattern and indicate it to the users "in some way" If that's an Open standard, and if that browser extension is Open Source, then anyone who wants to avoid that can mess with the final image until the Open Source free standard that everyone is using no longer detects the image. The only way this is viable is with DRM or a closed service; if it's a standard everyone follows, then circumventing it is trivial. The only way it would work is if it's shrouded in secrecy and attackers can't freely use red-team against the tool. These kinds of watermarks work when there's a very limited pool of people checking for the watermarks, they don't tell people who the watermarks are generated, and they don't tell people how to check for the watermarks. But that's not really useful for the current situation -- we don't want to further entrench these companies and we don't want it to be costly to check if an image is AI-generated. I don't think it's viable to do this without significantly curtailing user agency or designing a system that is fully opaque and inaccessible to most people.
- ke88y 3y ago
  Yeah, this is my general feeling about why this area is doomed. It's why I haven't bothered to write up a patent even though I had some good ideas a few years ago. Maybe that was a mistake since governments and corporate politicians are stupider than I assumed. If you have a central source of authority then the problem is totally trivial and the fact that the images are AI-generated (or not) is a complete red herring. If you don't have a central source of authority then any reasonable adversarial model makes the watermarking problem somewhere between very difficult and impossible. Detection from known models is still possible, at least for images. But that's not really watermarking per se.