9 ms·

Tackling the misinformation epidemic with “In Event of Moon Disaster”

82 points by MindGods 6y ago

mellosouls 6y ago
Very interesting and quite creepy in this specific context - genuinely enlightening as to wider possibilities. The actual faked Nixon speech is in the "Special Report" starting at 3.37 here (after a scene setting prelude): https://moondisaster.org/film https://moondisaster.org/film The original contingency letter from which the speech is derived: https://www.archives.gov/files/presidential-libraries/events/centennials/nixon/images/exhibit/rn100-6-1-2.pdf https://www.archives.gov/files/presidential-libraries/events...
- Angostura 6y ago
  Fascinating. It's good, but there are lots of little emphatic "head nods" that don't make sense in the context of the speech - unless that was just an odd mannerism of Nixon's.
  - owenmarshall 6y ago
    I wonder if a more recent subject would've been better. We all know the tics and habits of our current president, but Nixon was half a century ago and I wonder how many of us have seen his speeches in any real depth. Hell, play me Billy West reading Nixon as "Richard Nixon's head" from Futurama and I might buy it.
- teddyh 6y ago
  Obligatory xkcd: https://xkcd.com/1484/ https://xkcd.com/1484/
myself248 6y ago
The irony is that the people most susceptible to this kind of BS also don't believe we landed on the moon, so the effect may be somewhat muted...
- smabie 6y ago
  Everyone is susceptible to deep fakes.
gambler 6y ago
The very idea that deep fakes are somewhere near the top of the list of significant concerns for the media right now is in itself disinformation. And it seems to be very persistent. I keep seeing articles and videos about it all the time. Someone is pumping significant amount of money into this narrative. The irony is that all this "anti-disinformation" research will clearly be used to run better disinformation campaign if/when the primitive shit being used right now stops working.
- nsnow70 6y ago
  Care to elaborate?
  - gambler 6y ago
    Can you give me a single example of deepfakes being used to stage a real news event? Meanwhile, most of the population isn't even trained to recognize when videos are trimmed to start and stop at a conspicuously convenient moments, which is the most commonly used technique for creating fake narratives. Actual media literacy is at near zero, especially among the people who arrogantly consider themselves "informed".
    - nsnow70 6y ago
      Just to clarify, I'm not really picking a side but just was curious on your thoughts being explained further. Only thing I have to add is just because it's not a problem right this instant doesn't necessarily mean it can't be a problem in the future, right?
      - vkou 6y ago
        This argument is going in circles, though. Plain-old-media has been used, to great success, to manufacture news. Consider, nearly a decade ago, the BBC (And that's an outfit that many believe to have journalistic integrity) airing video footage of a protest in India, with people waving Indian flags, and claiming it to be video footage of anti-Ghadaffi protests in Libya, in the run-up to the Libya bombing campaigns. Millions of people have seen it, a few of them noticed that they are being fed a line of horseshit, and then they shrug their shoulders and go on with their day, because Gell-Mann amnesia is probably a real effect, and they didn't actually care too deeply about this. Worrying about deepfakes is a lot like worrying, around the invention of scissors that they could be used to stab people. Yes, they can be, but knives have been around for a lot longer, are easier to make, and work just as well. Going through the trouble of deepfakes, to convince the subset of people that will not be convinced by plain-old-multimedia-lies, but will be convinced by deepfake-multimedia-lies seems like a poor ROI maneuver - especially given that we live in a world where the number of people that can be convinced by plain-old-multimedia-lies is sufficient to achieve whatever end you seek.
jmount 6y ago
A tangent: the movie "The Landing" (2017) is a fun treatment of a non-existent Apollo 18 and disaster.
deleted 6y ago
[deleted]
prateek_mir 6y ago
A really good example to showcase the capability of the technology ! In context of India, where news channels have reneged from due diligence on evidently doctored videos, this is something which can have huge consequences.
camjohnson26 6y ago
Here’s the video: https://moondisaster.org/film https://moondisaster.org/film Not bad but the audio doesn’t sound quite right.
- ghaff 6y ago
  The thing is that you know it's a fake and are looking for clues that it's a fake. But for many purposes, it doesn't matter if you can fool careful study or expert analysis, it matters whether you can fool a quick glance from someone already predisposed to believe the contents who will retweet or share on facebook and move on.
  - LinuxBender 6y ago
    I agree. To add to this, I would suggest that this is good feedback to the people creating these and they will just see it as a challenge and / or enhancement request. Perhaps in time, even the best forensics won't be able to distinguish the difference. Maybe.
  - najarvg 6y ago
    Well said. Imagine a similar deepfake with a Nixon "confession" about having to stage the moon landing to make America look superior. You now suddenly have the next viral facebook or whatsapp forward for conspiracy theorists who are fully convinced of their beliefs.
    - rightbyte 6y ago
      Ye but it has been doable with modern movie production technology for some time without any fancy AI. The biggest difference is the price drop of it?
      - dogma1138 6y ago
        Accessibility and scalability, get a few trained models and you not only can produce the original deepfake but also a lot of convincing identities and accounts to spread it around. This is the same issue as RC va Drones, you could’ve strapped a pound of C4 to an RC flyer for decades but it was never as easy to do as with drones today, the skills, availability and reliability of consumer drones even the cheapest ones from Aliexpress allowed non-state actors to weaponize them with ease.
bsenftner 6y ago
15 years ago I created a "pre-deep fakes" automated actor replacement visual effects production pipeline. My ambition was to create "Personalized Advertising" where ordinary people would appear in video advertising for desirable products like film trailers, and various other products and services that typically have a celebrity spokesperson - the basic idea was the celebrity spokesperson would be "with you" in the advert explaining how great the product was while you are depicted nodding and enjoying it. The system worked, and still would work if I un-mothballed it. I even globally patented the system, with an ungodly expense doing so: https://patents.justia.com/patent/7460731 https://patents.justia.com/patent/7460731 However, nobody believed what I was pitching 15 years ago was possible. After I'd demonstrate a scaled down implementation, I'd get interest and an investment pool would start to form. Then one of the investors would realize the tech could be used for porn, and then no matter how I explained the economics of failure such an application would cause, their stupid dicks would take over and I could not get them to realize "porn with anyone inserted" is a lawsuit engine and not a sell-able product. While pursuing this, I became extremely jaded about what people think occurs in a film production, and the time and expense necessary to add visual effects to any media. The majority of angels and VC I pitched were flabbergasted at what current media and VFX productions are like, and their current level of expense and sophistication. My proposing a fully automated incarnation struck them as complete fantasy. Yet I had a working implementation. Slowly I'd convince them of the possibility... and then they'd get fixated on porn, ignoring the fact that there is no way to make money producing deep fake porn. I eventually went bankrupt and left media production entirely. After creating my fully automated VFX pipeline, working in VFX without my automation tools drove me nuts. I work in facial recognition now.
quacked 6y ago
Forget faking news events. What about petty crime? The cops get sent a video of you defacing property. Your phone data and several eyewitnesses tie you to the area. How do you escape, without a blanket ban on video evidence? If there's a blanket ban on video evidence, what if you have video proof that you were somewhere else?
- poorman 6y ago
  There are going to be people making a lot of money by giving expert testimony for a living.
- neixidbeksoxyd 6y ago
  If this becomes a problem, I think cameras and smartphones would be upgraded with a way to certify they are real videos. I have no idea how, and it would likely have to keep evolving, but if the financial incentives are there I'm sure someone will figure it out. Maybe the video gets signed by the phone and can only be verified by that device? This comment was generated by GPT-3
  - JoshuaDavid 6y ago
    Flash the flashlight in a sequence corresponding to the most recent hash on the bitcoin blockchain, then take a hash of the video and send 0.00000001 BTC to that address from a wallet associated with your personal identity? Would pretty much prove that a specific person recorded this video between two fairly close together times, and the blinking lights would probably do things with shadows that deepfakes aren't great at replicating (and retouching the video to fix those artifacts would make the hash not match).
ivanhoe 6y ago
For some reason shirt collars seem to be the weakest point on all deep fakes??
- acdanger 6y ago
  I noticed that, too. I kept noticing the chin skin tone "smudging" over the shirt collar throughout the video.
- exabrial 6y ago
  I noticed this too! I'm guessing state-level actors would probably review for this stuff and fix in post using existing editing tools.
Press2forEN 6y ago
There seems to be this hopeful belief out there that if only indisputably correct information was available to the general public, at long last they would finally support the correct policies and elect the correct leaders. I think that is a false premise.
- troughway 6y ago
  And yet even on HN this is disputed with such frequency as to warrant raising eyebrows as to what kind of mess we're walking into. Book burners are with us today, and they're embedded within the "cancel culture" and every online upvote/downvote system.
shadowprofile77 6y ago
Cute and well made job. I suspected that the video simply took a standard Nixon televised speech and used CGI to remake just the lower half of his face (thus the odd head movements even though his words and lips synced correctly, and if you look closely you can see that the lower part of his face looks somehow "cleaner" than the upper half), what stumped me a bit though was the audio. His voice sounds odd but recognizably his own, especially since Nixon had a rather unique voice. so I thought maybe they cut together words he'd said in different recorded contexts and digitally modulated the audio to construct a speech with an even tone, but no, they actually faked his voice, which impresses me more than the face CGI.
- chrisdalke 6y ago
  I don't think they are using conventional CGI techniques to "remake" his mouth/lips and lower face, if that is what you are suggesting. Seems like the dialog replacement is using a technology called VDR: https://www.fxguide.com/quicktakes/cannyai-vdr-face-replacement-as-a-service/ https://www.fxguide.com/quicktakes/cannyai-vdr-face-replacem.... There's a demo video on that page and some more details about the technique.
  - shadowprofile77 6y ago
    Sorry, maybe I'm conflating terminology, but I was referring to the use of AI and digital editing to reconstruct the lower part of his face so that it looks real while moving in ways that are false and say things he didn't actually say. This is what they did in some way at least, no?
    - chrisdalke 6y ago
      Yeah, I see your point, AI-generated CGI is still CGI!
DubiousPusher 6y ago
> an Information Ecosystem at Risk At Risk? Where have these people been for the last 40 years? I don't want to say the authenticity of information doesn't matter. But what clearly matters much more is collective trust in information institutions and from that standpoint the ecosystem has already utterly failed.
foldr 6y ago
To be honest, you could probably get more convincing results than this just by using spaghetti-Western-style dubbing of existing footage.
deleted 6y ago
[deleted]
14 6y ago
I am very curious how the legal system will handle deep fakes as they progress to perfect quality that even forensics can not tell.
- Nasrudith 6y ago
  There is already a tool in place that handles much of it, chain of custody for evidence. The video evidence would need corroborating evidence to "pin it down" essentially. Just showing say Hillary Clinton making infant stew wouldn't cut it without a chain of evidence proving there was a location, time, that a camera would actually be there, and there is some actual physical evidence of her murderous cannibalism.

82 points by MindGods 6y ago

Very interesting and quite creepy in this specific context - genuinely enlightening as to wider possibilities. The actual faked Nixon speech is in the "Special Report" starting at 3.37 here (after a scene setting prelude): https://moondisaster.org/film https://moondisaster.org/film The original contingency letter from which the speech is derived: https://www.archives.gov/files/presidential-libraries/events/centennials/nixon/images/exhibit/rn100-6-1-2.pdf https://www.archives.gov/files/presidential-libraries/events...

Angostura 6y ago
Fascinating. It's good, but there are lots of little emphatic "head nods" that don't make sense in the context of the speech - unless that was just an odd mannerism of Nixon's.
- owenmarshall 6y ago
  I wonder if a more recent subject would've been better. We all know the tics and habits of our current president, but Nixon was half a century ago and I wonder how many of us have seen his speeches in any real depth. Hell, play me Billy West reading Nixon as "Richard Nixon's head" from Futurama and I might buy it.
teddyh 6y ago
Obligatory xkcd: https://xkcd.com/1484/ https://xkcd.com/1484/

myself248 6y ago

The irony is that the people most susceptible to this kind of BS also don't believe we landed on the moon, so the effect may be somewhat muted...

smabie 6y ago
Everyone is susceptible to deep fakes.

gambler 6y ago

The very idea that deep fakes are somewhere near the top of the list of significant concerns for the media right now is in itself disinformation. And it seems to be very persistent. I keep seeing articles and videos about it all the time. Someone is pumping significant amount of money into this narrative. The irony is that all this "anti-disinformation" research will clearly be used to run better disinformation campaign if/when the primitive shit being used right now stops working.

nsnow70 6y ago
Care to elaborate?
- gambler 6y ago
  Can you give me a single example of deepfakes being used to stage a real news event? Meanwhile, most of the population isn't even trained to recognize when videos are trimmed to start and stop at a conspicuously convenient moments, which is the most commonly used technique for creating fake narratives. Actual media literacy is at near zero, especially among the people who arrogantly consider themselves "informed".
  - nsnow70 6y ago
    Just to clarify, I'm not really picking a side but just was curious on your thoughts being explained further. Only thing I have to add is just because it's not a problem right this instant doesn't necessarily mean it can't be a problem in the future, right?
    - vkou 6y ago
      This argument is going in circles, though. Plain-old-media has been used, to great success, to manufacture news. Consider, nearly a decade ago, the BBC (And that's an outfit that many believe to have journalistic integrity) airing video footage of a protest in India, with people waving Indian flags, and claiming it to be video footage of anti-Ghadaffi protests in Libya, in the run-up to the Libya bombing campaigns. Millions of people have seen it, a few of them noticed that they are being fed a line of horseshit, and then they shrug their shoulders and go on with their day, because Gell-Mann amnesia is probably a real effect, and they didn't actually care too deeply about this. Worrying about deepfakes is a lot like worrying, around the invention of scissors that they could be used to stab people. Yes, they can be, but knives have been around for a lot longer, are easier to make, and work just as well. Going through the trouble of deepfakes, to convince the subset of people that will not be convinced by plain-old-multimedia-lies, but will be convinced by deepfake-multimedia-lies seems like a poor ROI maneuver - especially given that we live in a world where the number of people that can be convinced by plain-old-multimedia-lies is sufficient to achieve whatever end you seek.

jmount 6y ago

A tangent: the movie "The Landing" (2017) is a fun treatment of a non-existent Apollo 18 and disaster.

deleted 6y ago

[deleted]

prateek_mir 6y ago

A really good example to showcase the capability of the technology ! In context of India, where news channels have reneged from due diligence on evidently doctored videos, this is something which can have huge consequences.

camjohnson26 6y ago

Here’s the video: https://moondisaster.org/film https://moondisaster.org/film Not bad but the audio doesn’t sound quite right.

ghaff 6y ago
The thing is that you know it's a fake and are looking for clues that it's a fake. But for many purposes, it doesn't matter if you can fool careful study or expert analysis, it matters whether you can fool a quick glance from someone already predisposed to believe the contents who will retweet or share on facebook and move on.
- LinuxBender 6y ago
  I agree. To add to this, I would suggest that this is good feedback to the people creating these and they will just see it as a challenge and / or enhancement request. Perhaps in time, even the best forensics won't be able to distinguish the difference. Maybe.
- najarvg 6y ago
  Well said. Imagine a similar deepfake with a Nixon "confession" about having to stage the moon landing to make America look superior. You now suddenly have the next viral facebook or whatsapp forward for conspiracy theorists who are fully convinced of their beliefs.
  - rightbyte 6y ago
    Ye but it has been doable with modern movie production technology for some time without any fancy AI. The biggest difference is the price drop of it?
    - dogma1138 6y ago
      Accessibility and scalability, get a few trained models and you not only can produce the original deepfake but also a lot of convincing identities and accounts to spread it around. This is the same issue as RC va Drones, you could’ve strapped a pound of C4 to an RC flyer for decades but it was never as easy to do as with drones today, the skills, availability and reliability of consumer drones even the cheapest ones from Aliexpress allowed non-state actors to weaponize them with ease.

bsenftner 6y ago

15 years ago I created a "pre-deep fakes" automated actor replacement visual effects production pipeline. My ambition was to create "Personalized Advertising" where ordinary people would appear in video advertising for desirable products like film trailers, and various other products and services that typically have a celebrity spokesperson - the basic idea was the celebrity spokesperson would be "with you" in the advert explaining how great the product was while you are depicted nodding and enjoying it. The system worked, and still would work if I un-mothballed it. I even globally patented the system, with an ungodly expense doing so: https://patents.justia.com/patent/7460731 https://patents.justia.com/patent/7460731 However, nobody believed what I was pitching 15 years ago was possible. After I'd demonstrate a scaled down implementation, I'd get interest and an investment pool would start to form. Then one of the investors would realize the tech could be used for porn, and then no matter how I explained the economics of failure such an application would cause, their stupid dicks would take over and I could not get them to realize "porn with anyone inserted" is a lawsuit engine and not a sell-able product. While pursuing this, I became extremely jaded about what people think occurs in a film production, and the time and expense necessary to add visual effects to any media. The majority of angels and VC I pitched were flabbergasted at what current media and VFX productions are like, and their current level of expense and sophistication. My proposing a fully automated incarnation struck them as complete fantasy. Yet I had a working implementation. Slowly I'd convince them of the possibility... and then they'd get fixated on porn, ignoring the fact that there is no way to make money producing deep fake porn. I eventually went bankrupt and left media production entirely. After creating my fully automated VFX pipeline, working in VFX without my automation tools drove me nuts. I work in facial recognition now.

quacked 6y ago

Forget faking news events. What about petty crime? The cops get sent a video of you defacing property. Your phone data and several eyewitnesses tie you to the area. How do you escape, without a blanket ban on video evidence? If there's a blanket ban on video evidence, what if you have video proof that you were somewhere else?

poorman 6y ago
There are going to be people making a lot of money by giving expert testimony for a living.
neixidbeksoxyd 6y ago
If this becomes a problem, I think cameras and smartphones would be upgraded with a way to certify they are real videos. I have no idea how, and it would likely have to keep evolving, but if the financial incentives are there I'm sure someone will figure it out. Maybe the video gets signed by the phone and can only be verified by that device? This comment was generated by GPT-3
- JoshuaDavid 6y ago
  Flash the flashlight in a sequence corresponding to the most recent hash on the bitcoin blockchain, then take a hash of the video and send 0.00000001 BTC to that address from a wallet associated with your personal identity? Would pretty much prove that a specific person recorded this video between two fairly close together times, and the blinking lights would probably do things with shadows that deepfakes aren't great at replicating (and retouching the video to fix those artifacts would make the hash not match).

ivanhoe 6y ago

For some reason shirt collars seem to be the weakest point on all deep fakes??

acdanger 6y ago
I noticed that, too. I kept noticing the chin skin tone "smudging" over the shirt collar throughout the video.
exabrial 6y ago
I noticed this too! I'm guessing state-level actors would probably review for this stuff and fix in post using existing editing tools.

Press2forEN 6y ago

There seems to be this hopeful belief out there that if only indisputably correct information was available to the general public, at long last they would finally support the correct policies and elect the correct leaders. I think that is a false premise.

troughway 6y ago
And yet even on HN this is disputed with such frequency as to warrant raising eyebrows as to what kind of mess we're walking into. Book burners are with us today, and they're embedded within the "cancel culture" and every online upvote/downvote system.

shadowprofile77 6y ago

Cute and well made job. I suspected that the video simply took a standard Nixon televised speech and used CGI to remake just the lower half of his face (thus the odd head movements even though his words and lips synced correctly, and if you look closely you can see that the lower part of his face looks somehow "cleaner" than the upper half), what stumped me a bit though was the audio. His voice sounds odd but recognizably his own, especially since Nixon had a rather unique voice. so I thought maybe they cut together words he'd said in different recorded contexts and digitally modulated the audio to construct a speech with an even tone, but no, they actually faked his voice, which impresses me more than the face CGI.

chrisdalke 6y ago
I don't think they are using conventional CGI techniques to "remake" his mouth/lips and lower face, if that is what you are suggesting. Seems like the dialog replacement is using a technology called VDR: https://www.fxguide.com/quicktakes/cannyai-vdr-face-replacement-as-a-service/ https://www.fxguide.com/quicktakes/cannyai-vdr-face-replacem.... There's a demo video on that page and some more details about the technique.
- shadowprofile77 6y ago
  Sorry, maybe I'm conflating terminology, but I was referring to the use of AI and digital editing to reconstruct the lower part of his face so that it looks real while moving in ways that are false and say things he didn't actually say. This is what they did in some way at least, no?
  - chrisdalke 6y ago
    Yeah, I see your point, AI-generated CGI is still CGI!

DubiousPusher 6y ago

> an Information Ecosystem at Risk At Risk? Where have these people been for the last 40 years? I don't want to say the authenticity of information doesn't matter. But what clearly matters much more is collective trust in information institutions and from that standpoint the ecosystem has already utterly failed.

foldr 6y ago

To be honest, you could probably get more convincing results than this just by using spaghetti-Western-style dubbing of existing footage.

deleted 6y ago

[deleted]

14 6y ago

I am very curious how the legal system will handle deep fakes as they progress to perfect quality that even forensics can not tell.

Nasrudith 6y ago
There is already a tool in place that handles much of it, chain of custody for evidence. The video evidence would need corroborating evidence to "pin it down" essentially. Just showing say Hillary Clinton making infant stew wouldn't cut it without a chain of evidence proving there was a location, time, that a camera would actually be there, and there is some actual physical evidence of her murderous cannibalism.