6 ms·

Why? Don’t do this. Society is built on an implicit assumption of trust. You will erode the foundation that enables any success you might have in the short term

Why? Don’t do this. Society is built on an implicit assumption of trust. You will erode the foundation that enables any success you might have in the short term.

xorgun 5mo ago
[dead]
vasco 5mo ago
How is a hidden watermark that only police and providers can read better than a hidden watermark that the people can also read if they go out of their way to setup some project like this? Also your argument makes zero sense because bad guys will do this anyway. At least there's also an open source project for normal people, and it's interesting from a stenography standpoint.
- digitaltrees 5mo ago
  Watermarks exists to create a chain of custody or attribution that can be used to establish culpability. Just like the government requires printer companies to add a dot matrix watermark to printed pages to prevent currency forgery. Making untraceable assets means parties can’t be held liable for harms. Just because a bad actor might poison a toddler doesn’t mean you should sell them the arsenic. Just because the effects of arsenic on the neurons of a brain are interesting means you should feed them to a toddler and watch the effects unfold. I support the legal freedom to pursue any idea, but we should also mentor our colleagues about the consequences of our projects and avoid unnecessary harm.
  - vasco 5mo ago
    What is it with people that take your position that they always need to bring children to the discussion? Where's the watermark in an IKEA knife?
    - digitaltrees 5mo ago
      Is an ikea knife infinitely replicable, distributable and capable of undermining the entire social fabric of a community? No. so these are not the same category of threat potential. Further, an ikea knife used in a crime can be traced back to the criminal through other evidence like finger prints, surveillance cameras, purchase receipts etc. Arguments that have a emotional component, especially related to our evolutionary instinct to take care of kids has a way of avoiding the rationalizations people adopt to justify otherwise indefensible behavior. You also dont do yourself any favors in a bad argument that lumps me in with some unknown group of people rather tan responding to my argument.
  - xeyownt 5mo ago
    No. All watermarks are cures that are worse than the diseases they are trying to prevent (and never achieved to even cure anything). Watermark is a blatant violation of legitimate privacy expectations, while not preventing anything because they can be easily removed. So all the honest usages are punished, while illegitimate ones are not caught.
    - digitaltrees 5mo ago
      A water mark that identifies you personally is a different prospect than one that merely identifies the image as AI generated. I am with you on opposing a watermark that identifies a person. I am not with you in saying all watermarks are a violation of privacy or that they result in punishment of honest use. I don’t see how those follow.
jauer 5mo ago
Information is default low-trust unless you have reason to extend trust to the source and that's been the case for thousands of years, if not the entirety of human existence. We now have the tools to increase trust in specific information, for example: by signing images that need high trust for things like news reporting using camera hardware root of trust with time and geo stamping. If signatures are removed, that's back to a default low-trust state.
- devsda 5mo ago
  It is best not to extend blanket trust to a specific source at all. That is how we ended up with the situation where "reputed" media organizations peddle daily lies or selective truths that are useful to their benefactors.
- digitaltrees 5mo ago
  I don’t want kidnapping scammers to have easy to fake image generators that are untraceable. I don’t want eBay scammers to have that (sometimes I will ask the seller to take a picture with specific information to confirm a listing is credible). I could site lots more examples
egeozcan 5mo ago
IMHO you cannot base your trust on something that is so easy to work around. Trust should be between people, not a person and some data. So showing how easy to generate "false" data, this makes it more obvious for people focus on other people. Trusting people makes life much easier in my experience, while focusing on data, again in my experience, is a game of cat and mouse.
- goatlover 5mo ago
  The Kentucky primary had $1.7 million spent on deepfake political ads which were seen 49 million times. Don't know how much it effected the result, but it's not a good sign of where things are headed.
- InsideOutSanta 5mo ago
  > Trusting people makes life much easier in my experience Sure, but how do you apply that to a society at large where powerful people are interested in making everybody distrust all reliable sources of information? AI watermarks are no panacea, but at least they are a clear signal of what not to trust.
  - survirtual 5mo ago
    AI watermarks empower elite / those with resources and disempowers the common person. Only people with resources will be allowed to make content that is AI generated passed off as real. Pandora's box is open. Instead of making a multi-tiered privileged society, we need to fundamentally restructure society to adapt. Before that restructuring occurs it is critical to keep the playing field level. These are not tools that should be controlled by a minority authority, they are far too dangerous.
    - digitaltrees 5mo ago
      That’s a reasonable argument. But why should we tolerate the two tier system? Why not enforce a restriction on all images? Why do we have to accept someone is above the law?
basch 5mo ago
The distortion is shockingly visible in images. Especially with any amount of generational iteration https://streamable.com/9x3s4r https://streamable.com/9x3s4r That said, this tool is incredibly lossy, garbling text, completely changing shapes. It also fails to remove the new gemini spark / placement that moved since yesterday.
antiloper 5mo ago
> You will erode the foundation that enables any success you might have in the short term. It's too late already. We live in a post-trust society now.
- digitaltrees 5mo ago
  But why make it worse and easier to avoid detection?
bsza 5mo ago
Would you prefer if someone did the same thing and kept it to themselves (or sold it to the highest bidder)? I think knowing it exists is better than not knowing it exists.
- digitaltrees 5mo ago
  No. I would prefer we, as a community, don’t build this. I am under no illusion that I will persuade all people but I feel clarity that it is morally and practically unwise to build a tool like this at all and especially unwise to publish it as a freely available open source project
  - bsza 5mo ago
    Nothing morally wrong about finding an exploit in a system, it's what allows you to make it more secure in the future. Perhaps the most ethical course of action would have been to disclose this to Google/OAI first (which I don't know whether or not has happened), but I find that optional in this case since this isn't really a vulnerability in the conventional sense.
    - digitaltrees 5mo ago
      Finding an exploit with the intent to patch it is different that finding the exploit and using it for personal gain which, in turn, is different than finding the exploit and publishing it for open source ecosystem use. It’s kind like if I took a picture off of your facebook with your keychain, and used it to make a copy of your house key. You’d probably prefer I reached out and told you to take down the picture instead of creating a template of the key for anyone to download and make a copy along with your home address.
probably_wrong 5mo ago
I'd like to frame this differently: watermarking is a (weaker) form of DRM, and DRM has never worked in favor of the users. I know of at least one music GenAI service whose ToS forbid me from using their productions in ways that are incompatible with my local rights. If Google decides, as they did with YouTube, that they'll enforce this company's watermark even though I have a right to use their results then I've passively accepted that a (foreign) company decides which of my legal rights I can exercise. For a more pessimistic outlook: for the GenAI companies that work for the military, are those pictures also watermarked? Because if watermarks only apply to one group then the "implicit trust" argument doesn't work. I'm in favor of watermarking the same way I am in favor of paying artists for their work. But just as with youtube-dl and DVD decryption libraries, tools like these are necessary to level the playing field.
- digitaltrees 5mo ago
  I don’t think the military should be allowed to generate fake images. I do think there should be international standards on AI because the social harm is universal and dangerous. Just like we don’t allow chemical weapons etc.
hathym 5mo ago
lol
- digitaltrees 5mo ago
  The electricity used to send the http request, run the lisp code on the server, save it to whatever database Paul Graham used to build HN and render it on my display could have been put to better use that your comment. Or mine. :)
b65e8bee43c2ed0 5mo ago
>Why? Don’t do this. Society is built on an implicit assumption of trust. You will erode the foundation that enables any success you might have in the short term. would you say the same thing about a ROT13 breaker, or would you recognize how laughably naive that sounds?
- digitaltrees 5mo ago
  I have no idea what a Rot13 breaker is and dont care that you think advocating for an ethical posture in what the tech industry builds is naive. Its naive to think bullies wont be in a kids playground. Its also naive to think we just need to accept that rather than creating norms around the type of play that is acceptable so that the playground functions. AI generated images have never existed before, they will break our ability to use the digital tools we have built society on if we let them. Ensuring they can be identified and have attribution tracing data embedded is a reasonable step to prevent abuse.
__MatrixMan__ 5mo ago
Society was build on that assumption, it didn't last. Now we have to build a new one where we put in the work of explicitly trusting each other. Preserving the illusion that that assumption is still useful only helps the people who are exploiting it.

Why? Don’t do this. Society is built on an implicit assumption of trust. You will erode the foundation that enables any success you might have in the short term.

xorgun 5mo ago

[dead]

vasco 5mo ago

How is a hidden watermark that only police and providers can read better than a hidden watermark that the people can also read if they go out of their way to setup some project like this? Also your argument makes zero sense because bad guys will do this anyway. At least there's also an open source project for normal people, and it's interesting from a stenography standpoint.

digitaltrees 5mo ago
Watermarks exists to create a chain of custody or attribution that can be used to establish culpability. Just like the government requires printer companies to add a dot matrix watermark to printed pages to prevent currency forgery. Making untraceable assets means parties can’t be held liable for harms. Just because a bad actor might poison a toddler doesn’t mean you should sell them the arsenic. Just because the effects of arsenic on the neurons of a brain are interesting means you should feed them to a toddler and watch the effects unfold. I support the legal freedom to pursue any idea, but we should also mentor our colleagues about the consequences of our projects and avoid unnecessary harm.
- vasco 5mo ago
  What is it with people that take your position that they always need to bring children to the discussion? Where's the watermark in an IKEA knife?
  - digitaltrees 5mo ago
    Is an ikea knife infinitely replicable, distributable and capable of undermining the entire social fabric of a community? No. so these are not the same category of threat potential. Further, an ikea knife used in a crime can be traced back to the criminal through other evidence like finger prints, surveillance cameras, purchase receipts etc. Arguments that have a emotional component, especially related to our evolutionary instinct to take care of kids has a way of avoiding the rationalizations people adopt to justify otherwise indefensible behavior. You also dont do yourself any favors in a bad argument that lumps me in with some unknown group of people rather tan responding to my argument.
- xeyownt 5mo ago
  No. All watermarks are cures that are worse than the diseases they are trying to prevent (and never achieved to even cure anything). Watermark is a blatant violation of legitimate privacy expectations, while not preventing anything because they can be easily removed. So all the honest usages are punished, while illegitimate ones are not caught.
  - digitaltrees 5mo ago
    A water mark that identifies you personally is a different prospect than one that merely identifies the image as AI generated. I am with you on opposing a watermark that identifies a person. I am not with you in saying all watermarks are a violation of privacy or that they result in punishment of honest use. I don’t see how those follow.

jauer 5mo ago

Information is default low-trust unless you have reason to extend trust to the source and that's been the case for thousands of years, if not the entirety of human existence. We now have the tools to increase trust in specific information, for example: by signing images that need high trust for things like news reporting using camera hardware root of trust with time and geo stamping. If signatures are removed, that's back to a default low-trust state.

devsda 5mo ago
It is best not to extend blanket trust to a specific source at all. That is how we ended up with the situation where "reputed" media organizations peddle daily lies or selective truths that are useful to their benefactors.
digitaltrees 5mo ago
I don’t want kidnapping scammers to have easy to fake image generators that are untraceable. I don’t want eBay scammers to have that (sometimes I will ask the seller to take a picture with specific information to confirm a listing is credible). I could site lots more examples

egeozcan 5mo ago

IMHO you cannot base your trust on something that is so easy to work around. Trust should be between people, not a person and some data. So showing how easy to generate "false" data, this makes it more obvious for people focus on other people. Trusting people makes life much easier in my experience, while focusing on data, again in my experience, is a game of cat and mouse.

goatlover 5mo ago
The Kentucky primary had $1.7 million spent on deepfake political ads which were seen 49 million times. Don't know how much it effected the result, but it's not a good sign of where things are headed.
InsideOutSanta 5mo ago
> Trusting people makes life much easier in my experience Sure, but how do you apply that to a society at large where powerful people are interested in making everybody distrust all reliable sources of information? AI watermarks are no panacea, but at least they are a clear signal of what not to trust.
- survirtual 5mo ago
  AI watermarks empower elite / those with resources and disempowers the common person. Only people with resources will be allowed to make content that is AI generated passed off as real. Pandora's box is open. Instead of making a multi-tiered privileged society, we need to fundamentally restructure society to adapt. Before that restructuring occurs it is critical to keep the playing field level. These are not tools that should be controlled by a minority authority, they are far too dangerous.
  - digitaltrees 5mo ago
    That’s a reasonable argument. But why should we tolerate the two tier system? Why not enforce a restriction on all images? Why do we have to accept someone is above the law?

basch 5mo ago

The distortion is shockingly visible in images. Especially with any amount of generational iteration https://streamable.com/9x3s4r https://streamable.com/9x3s4r That said, this tool is incredibly lossy, garbling text, completely changing shapes. It also fails to remove the new gemini spark / placement that moved since yesterday.

antiloper 5mo ago

> You will erode the foundation that enables any success you might have in the short term. It's too late already. We live in a post-trust society now.

digitaltrees 5mo ago
But why make it worse and easier to avoid detection?

bsza 5mo ago

Would you prefer if someone did the same thing and kept it to themselves (or sold it to the highest bidder)? I think knowing it exists is better than not knowing it exists.

digitaltrees 5mo ago
No. I would prefer we, as a community, don’t build this. I am under no illusion that I will persuade all people but I feel clarity that it is morally and practically unwise to build a tool like this at all and especially unwise to publish it as a freely available open source project
- bsza 5mo ago
  Nothing morally wrong about finding an exploit in a system, it's what allows you to make it more secure in the future. Perhaps the most ethical course of action would have been to disclose this to Google/OAI first (which I don't know whether or not has happened), but I find that optional in this case since this isn't really a vulnerability in the conventional sense.
  - digitaltrees 5mo ago
    Finding an exploit with the intent to patch it is different that finding the exploit and using it for personal gain which, in turn, is different than finding the exploit and publishing it for open source ecosystem use. It’s kind like if I took a picture off of your facebook with your keychain, and used it to make a copy of your house key. You’d probably prefer I reached out and told you to take down the picture instead of creating a template of the key for anyone to download and make a copy along with your home address.

probably_wrong 5mo ago

I'd like to frame this differently: watermarking is a (weaker) form of DRM, and DRM has never worked in favor of the users. I know of at least one music GenAI service whose ToS forbid me from using their productions in ways that are incompatible with my local rights. If Google decides, as they did with YouTube, that they'll enforce this company's watermark even though I have a right to use their results then I've passively accepted that a (foreign) company decides which of my legal rights I can exercise. For a more pessimistic outlook: for the GenAI companies that work for the military, are those pictures also watermarked? Because if watermarks only apply to one group then the "implicit trust" argument doesn't work. I'm in favor of watermarking the same way I am in favor of paying artists for their work. But just as with youtube-dl and DVD decryption libraries, tools like these are necessary to level the playing field.

digitaltrees 5mo ago
I don’t think the military should be allowed to generate fake images. I do think there should be international standards on AI because the social harm is universal and dangerous. Just like we don’t allow chemical weapons etc.

hathym 5mo ago

lol

digitaltrees 5mo ago
The electricity used to send the http request, run the lisp code on the server, save it to whatever database Paul Graham used to build HN and render it on my display could have been put to better use that your comment. Or mine. :)

b65e8bee43c2ed0 5mo ago

>Why? Don’t do this. Society is built on an implicit assumption of trust. You will erode the foundation that enables any success you might have in the short term. would you say the same thing about a ROT13 breaker, or would you recognize how laughably naive that sounds?

digitaltrees 5mo ago
I have no idea what a Rot13 breaker is and dont care that you think advocating for an ethical posture in what the tech industry builds is naive. Its naive to think bullies wont be in a kids playground. Its also naive to think we just need to accept that rather than creating norms around the type of play that is acceptable so that the playground functions. AI generated images have never existed before, they will break our ability to use the digital tools we have built society on if we let them. Ensuring they can be identified and have attribution tracing data embedded is a reasonable step to prevent abuse.

__MatrixMan__ 5mo ago

Society was build on that assumption, it didn't last. Now we have to build a new one where we put in the work of explicitly trusting each other. Preserving the illusion that that assumption is still useful only helps the people who are exploiting it.