4 ms·

I actually just published a guide on how you can help ensure that ("deepfake") tools that you make are less likely to end up misused like this: https://www.tech

by avivo 7y ago

I actually just published a guide on how you can help ensure that ("deepfake") tools that you make are less likely to end up misused like this: https://www.technologyreview.com/s/614912/ethical-deepfake-tools-a-manifesto/ https://www.technologyreview.com/s/614912/ethical-deepfake-t... Curious what folks think. I also just posted it to /newest if you want to respond there.

catalogia 7y ago
Is anybody old enough to remember if people once had analogous reactions to software like photoshop?
- hoistbypetard 7y ago
  While I don't recall any specific article like this one published, I do remember people writing about similar worries in the mid '90s.
- michaelbuckbee 7y ago
  I am (mid-forties) and started using Photoshop around version 2. People definitely worried about the integrity of photographic evidence. I think what's different now is the combination of much more powerful cheaper tools (those fake GAN generated faces are likely taken from one of the sites making them for free) vs Photoshop at the time was thousands of inflation-adjusted dollars. This is coupled with a much more efficient peer to peer and unvalidated news system (which there's much to like), but consider Dan Rather. He was forced out of CBS Evening News and producers were fired for having been fooled by (not producing) digitally forged memos for a story critical of George W. Bush. [1] Contrast that to whatever group is putting up fake stories and accounts of Facebook, there are no repercussions. 1 - https://en.wikipedia.org/wiki/Dan_Rather#Killian_documents https://en.wikipedia.org/wiki/Dan_Rather#Killian_documents
  - btilly 7y ago
    What is even crazier about the Dan Rather story is that the story about how George W. Bush avoided Vietnam due to political manipulation and bribes was actually TRUE. Before it was ever reported in the USA it was reported in the UK by The Guardian among others. Dan Rather's team started their investigation with sources in hand which already constituted very strong evidence. Given that the story was true and Dan Rather's team was starting with solid evidence of it, Bush's team was at a disadvantage in discrediting the story. Their solution was to slip a clearly faked smoking gun that was so good that Rather would have to lead with it. After Rather lead with it, Bush's team was able to discredit Rather and get him fired. The result was that the story became toxic and nobody in the USA was willing to report the story. The result is rather amazing. By faking evidence for what really did happen, Bush's team was able to make everyone believe that it didn't happen! And was able to keep the story out of the US media!
    - catalogia 7y ago
      Part of the problem may have been the story didn't really resonate with the American public. In theory I guess I'm meant to hate cowards and hypocrites, but learning that Bush the Younger once feared war is almost humanizing.
    - nradov 7y ago
      That's an interesting conspiracy theory. Now where's the hard evidence?
      - btilly 7y ago
        I read it in one of Greg Palast's books. He reported on the story for the BBC, and knew what evidence Rather's team started with because he was one of the people who supplied it to them. You can also still find reporting on the story online from The Guardian if you look for it. See https://en.wikipedia.org/wiki/Greg_Palast https://en.wikipedia.org/wiki/Greg_Palast for more on the reporter that I named. And lest you think that this kind of dirty and underhanded trick would have been off limits, see https://en.wikipedia.org/wiki/Swift_Vets_and_POWs_for_Truth https://en.wikipedia.org/wiki/Swift_Vets_and_POWs_for_Truth for a more widely known story of the underhanded tricks used by Bush supporters in that particular election.
wayoutthere 7y ago
I'm not sure it even matters if the tools are used at all -- the mere "boogeyman" factor that it's possible will likely cause people to start discounting video evidence as real. The same happens with altered photos already. Nobody believes a photo anymore.
avivo 7y ago
Here is a summary of potential mitigations for those too busy to click: # Limiting users or use cases - User Vetting: Carefully vet those who can use a tool (Example: Synthesia only working with vetted enterprise clients.) - Usage Constraints: Limit the synthesis possibilities. (Example: Humen is providing only a limited set of movements for generated videos.) # Discouraging malicious use of flexible and public tools - Clear disclosure: Ensuring and requiring that manipulated outputs are clearly labeled (including via metadata). - Consent protection: Requiring the consent of those being impersonated. - Detection friendliness: Supporting those working on detection and not seeking to fool detection systems. - Hidden watermarks: Embedding data through robust watermarks. - Usage logs: Storing information about usage and media outputs. - Use restrictions: Enforcing contracts that prohibit forms of violative or malicious use. Not all apply to every system of course. The article has more detail!
blotter_paper 7y ago
> Curious what folks think. Personal take: I value open source software, and while you do note that some of your suggestions can be implemented in open source while hoping that bad actors don't take the time to remove them you also suggest (especially in the other article that elaborates on when to release code) that some code simply shouldn't be shared with the public. I hope you lose the culture war, and we find other ways to deal with this issue. The people I trust least are the ones most capable of forging evidence, and I hope that proliferation of this technology you fear will cause us to rethink our standards of trust and evidence altogether; signature or it didn't happen.

by avivo 7y ago

catalogia 7y ago

Is anybody old enough to remember if people once had analogous reactions to software like photoshop?

hoistbypetard 7y ago
While I don't recall any specific article like this one published, I do remember people writing about similar worries in the mid '90s.
michaelbuckbee 7y ago
I am (mid-forties) and started using Photoshop around version 2. People definitely worried about the integrity of photographic evidence. I think what's different now is the combination of much more powerful cheaper tools (those fake GAN generated faces are likely taken from one of the sites making them for free) vs Photoshop at the time was thousands of inflation-adjusted dollars. This is coupled with a much more efficient peer to peer and unvalidated news system (which there's much to like), but consider Dan Rather. He was forced out of CBS Evening News and producers were fired for having been fooled by (not producing) digitally forged memos for a story critical of George W. Bush. [1] Contrast that to whatever group is putting up fake stories and accounts of Facebook, there are no repercussions. 1 - https://en.wikipedia.org/wiki/Dan_Rather#Killian_documents https://en.wikipedia.org/wiki/Dan_Rather#Killian_documents
- btilly 7y ago
  What is even crazier about the Dan Rather story is that the story about how George W. Bush avoided Vietnam due to political manipulation and bribes was actually TRUE. Before it was ever reported in the USA it was reported in the UK by The Guardian among others. Dan Rather's team started their investigation with sources in hand which already constituted very strong evidence. Given that the story was true and Dan Rather's team was starting with solid evidence of it, Bush's team was at a disadvantage in discrediting the story. Their solution was to slip a clearly faked smoking gun that was so good that Rather would have to lead with it. After Rather lead with it, Bush's team was able to discredit Rather and get him fired. The result was that the story became toxic and nobody in the USA was willing to report the story. The result is rather amazing. By faking evidence for what really did happen, Bush's team was able to make everyone believe that it didn't happen! And was able to keep the story out of the US media!
  - catalogia 7y ago
    Part of the problem may have been the story didn't really resonate with the American public. In theory I guess I'm meant to hate cowards and hypocrites, but learning that Bush the Younger once feared war is almost humanizing.
  - nradov 7y ago
    That's an interesting conspiracy theory. Now where's the hard evidence?
    - btilly 7y ago
      I read it in one of Greg Palast's books. He reported on the story for the BBC, and knew what evidence Rather's team started with because he was one of the people who supplied it to them. You can also still find reporting on the story online from The Guardian if you look for it. See https://en.wikipedia.org/wiki/Greg_Palast https://en.wikipedia.org/wiki/Greg_Palast for more on the reporter that I named. And lest you think that this kind of dirty and underhanded trick would have been off limits, see https://en.wikipedia.org/wiki/Swift_Vets_and_POWs_for_Truth https://en.wikipedia.org/wiki/Swift_Vets_and_POWs_for_Truth for a more widely known story of the underhanded tricks used by Bush supporters in that particular election.

wayoutthere 7y ago

I'm not sure it even matters if the tools are used at all -- the mere "boogeyman" factor that it's possible will likely cause people to start discounting video evidence as real. The same happens with altered photos already. Nobody believes a photo anymore.

avivo 7y ago

Here is a summary of potential mitigations for those too busy to click: # Limiting users or use cases - User Vetting: Carefully vet those who can use a tool (Example: Synthesia only working with vetted enterprise clients.) - Usage Constraints: Limit the synthesis possibilities. (Example: Humen is providing only a limited set of movements for generated videos.) # Discouraging malicious use of flexible and public tools - Clear disclosure: Ensuring and requiring that manipulated outputs are clearly labeled (including via metadata). - Consent protection: Requiring the consent of those being impersonated. - Detection friendliness: Supporting those working on detection and not seeking to fool detection systems. - Hidden watermarks: Embedding data through robust watermarks. - Usage logs: Storing information about usage and media outputs. - Use restrictions: Enforcing contracts that prohibit forms of violative or malicious use. Not all apply to every system of course. The article has more detail!

blotter_paper 7y ago

> Curious what folks think. Personal take: I value open source software, and while you do note that some of your suggestions can be implemented in open source while hoping that bad actors don't take the time to remove them you also suggest (especially in the other article that elaborates on when to release code) that some code simply shouldn't be shared with the public. I hope you lose the culture war, and we find other ways to deal with this issue. The people I trust least are the ones most capable of forging evidence, and I hope that proliferation of this technology you fear will cause us to rethink our standards of trust and evidence altogether; signature or it didn't happen.