7 ms·

This is what's being accomplished by one amateur with a little bit of hardware. Imagine what will happen when you have professional services (legal or otherwise

by mabbo 9y ago

This is what's being accomplished by one amateur with a little bit of hardware. Imagine what will happen when you have professional services (legal or otherwise) making altered videos of whoever or whatever you want. Want court evidence of your wife cheating? Fake grainy hidden camera video. Want to discredit a politician? Here he is picking up a male escort and using racial slurs. These sorts of things could be used to start wars. Sure, deeper analysis might prove that this video isn't real, but that won't stop the immediate public reaction and response. The knee-jerk reaction to what the public sees is the end-goal. I'll bet good money that this technology plays a major role in future elections.

digi_owl 9y ago
> Want to discredit a politician? Here he is picking up a male escort and using racial slurs. Worked so well in a recent election... > These sorts of things could be used to start wars. Wars have been started on much flimsier pretexts, unless you are talking about getting the other side so irate that they fire first.
beisner 9y ago
I think we’ll probably have to move to a model where any security footage or images that is used in court must be digitally signed at the hardware level, with a verification process similar to how CAs work now. Of course, this will still be tamperable, but tampering would have to be much more premeditated and sophisticated.
- omeid2 9y ago
  The privacy implications of that would be scary. It seems on par with North Korea's Red Star Watermarking which has been dubbed as "dictator’s wet dream". https://qz.com/583311/north-koreas-dissident-tracking-computer-software-is-a-dictators-wet-dream/ https://qz.com/583311/north-koreas-dissident-tracking-comput...
- throwaway2048 9y ago
  that is absolutely never going to be workable because you can just film another video being played back.
  - nine_k 9y ago
    I can imagine watermarking at the light sensor level. It would need some very special provisions for compression, though.
    - striking 9y ago
      Well, but then you'd have to trust the light sensor manufacturer to provide secure keys. Remember when the NSA hacked into Gemalto and had a chance to steal its SIM card keys? However, that's still interesting; isn't it possible that each light sensor has certain watermark-like characteristics arising from the manufacturing process?
      - c22 9y ago
        Yes, fingerprinting of digital cameras has been a thing for at least a decade [https://www.eurekalert.org/pub_releases/2006-04/bu-bur041806.php https://www.eurekalert.org/pub_releases/2006-04/bu-bur041806...]
    - sametmax 9y ago
      Problem is, this way journalists won't have anonymous sources anymore and you will be able to track any citizen filming power abuses.
  - beisner 9y ago
    I mean, there are all kinds of ways to bound uncertainty. You’re right, you could film another camera, but that could be addressed separately. Digital signing would only prove that a video came directly from a particular camera’s sensors. To assert the veracity of a video, you’d then have to prove that particular camera was at the scene, and the light coming into the lens was authentic. Most court footage is incidental, not intentional, so the sort of doctoring you describe is unlikely in a majority of cases. Where it is likely, different measures could be taken to bound uncertainty. Separately, generating false images or videos for libel purposes would become more difficult; if a camera that has never been owned by a person is used to produce a signed video that is pornographic in nature, the authenticity of that video is immediately suspect. It would take a much more sophisticated attack to produce a mathematically convincing fake video.
paul7986 9y ago
And that’s why the use and mandating everyone uses their real and verified identity everywhere on the web is needed. You can still comment and post whatever anonymously but it won’t hold any weight!
- ameister14 9y ago
  That doesn't really make sense in this context; if someone is pretending to be you using your own face and voice, using real identities is kind of the point, isn't it?
  - paul7986 9y ago
    Ummm but if it’s not you the uploader thru a verified ID system (block chain maybe) is held liable immediately. Also by the time such a system is out in place the public will be even more suspect of believing what they see on the net as truth. Such is already happening with those 40 and below in terms of finally waking up to all the fakery on the web. Just saw on FB a video where a girl crosses her eyes and they get stuck. Majority of commenters below 40 notes it’s fake. Above 40 commented otherwise. Either we instate an ID system or the web becomes a joke no one believes. The latter I guess wouldn’t be a bad thing.
everdev 9y ago
While the technique does make fraudulent videos easier to make, I don't see this in and of itself as that fatal to public trust. Faking content to hurt someone has been around forever. We then learned not to trust gossip, not to trust uncorroborated sources, etc. I hope it might actually help people be more skeptical of what they're reading and seeing. And hopefully it will help news media be more skeptical of their sources as well. If that doesn't happen then this will be a disaster.
- phinnaeus 9y ago
  > We then learned not to trust gossip, not to trust uncorroborated sources, etc. Somewhat ironically, I feel a citation is needed here.
  - sverige 9y ago
    #MeToo
  - Izkata 9y ago
    Makes me think of the phrase "pics or it didn't happen". Granted it was a lot more common before photoshop-style software was everywhere.
- cyberpunk0 9y ago
  You overestimate people
- sametmax 9y ago
  > We then learned not to trust gossip, not to trust uncorroborated sources, etc. If it needed any more proof (than facebook, fox, ads, gov propaganda...) that this never happened, I think the Trump debacle should be final evidence this is fantasy.
- jaakl 9y ago
  The video by itself will be shortly a medium what noone believes as ground truth, just as a written word or a drawing is now. We should be able to construct better cameras with tamperproof footage. But not sure how, as with realtime editing we cannot trust trivial things like secure timestamping. Secure video blockchain anyone?
SanFranManDan 9y ago
All of this has been possible for the last century. For instance, at least in Russia you just get an actor that looks like the politician, film them doing some bad stuff and blackmail them. Pretty simple. We have had doctored photos/videos for as long as the mediums existed. If anything it will saturate and you wont be able to tell what is real or not. Most people will default to "fake" and have to rely on the trustfulness of the source. Just like now. We have photoshop and there isn't mass chaos.
- dctoedt 9y ago
  > Most people will default to "fake" and have to rely on the trustfulness of the source. Until faked video of the trusted source him- or herself appears.
chrisper 9y ago
These days you don't even seem to need any evidence. An allegation seems to be enough to get you dethroned.
Sir_Cmpwn 9y ago
How long will it take for this approach to be used to splice people into child pornography? The court of public opinion might sentence the victim to maiming or death.
- dredmorbius 9y ago
  Such things have been known to happen. Emmet Till. The Dutch de Witt brothers in 1672 are a notorious example. False testimony, witness, and evidence are all things. http://tywkiwdbi.blogspot.com/2010/09/corpses-of-de-witt-brothers.html http://tywkiwdbi.blogspot.com/2010/09/corpses-of-de-witt-bro...
jcfrei 9y ago
The problem I see with tech like this is that it muddies the water for everyone. I find it very strange that technologies like the internet appear to have created a world with more ambiguity and less transparency than we've had before - mostly because people now can find support for any kind of belief online. Some people are either willing to learn or have learned how to get information from reputation based systems and some will just go rogue and create their own reality. I hope democracy survives this shift: Because not only will we have to discuss the facts, now we will also have to discuss: What are the facts?
- toAnswerIt 9y ago
  I think this is a fallacy. the olden days were way worse. to put in presepective Carrots improving vision is propaganda created by the british yet see how many people believed it. Humans inventing and accepting fake beliefs to comfort themselves was never a new thing. Governments gaslighting its own citizens is communist party 101. People always want to believe that yesterday was better and blame their problems on the new.
  - gaius 9y ago
    Carrots improving vision is propaganda created by the british yet see how many people believed it. Not exactly propaganda, more misdirection, maskirovka our Russian comrades call it
- vertex-four 9y ago
  We've always had to discuss what the facts are - the extreme bias between newspapers and TV channels picking and choosing facts to provide to their users, even on the same news stories - news source A chooses not to provide background info, news source B chooses to provide background info from one primary source only, news source C chooses to provide background info from another source, news source D chooses to provide a general overview but with a slight bent towards assuming that its readers hold values X, Y and Z, etc etc. You can go back and read stories from the 1950s which have these issues and more. There has never been a time where all individuals involved in a democracy have understood all the facts. Popular politics has always been driven by newspapers and special interest groups. In my country, implementation of a certain policy has very clear correlation with a marked increase in suicide rates - the slow rollout of this specific policy across areas of the country provides an easy way to show this. Certain newspapers refuse to report on this - it doesn't match their ideology - after all, when pointed out, people holding this ideology will often point out that suicide is one's own decision and it's not the Government's problem.
jessaustin 9y ago
This just gives more power to "authority". If subjects can't believe their own eyes anymore, who will tell them what to think? NBC would be happy to take the job...
jokoon 9y ago
Well I disagree. There are many ways to disprove such video, and if the technology exists, we already know that it exists, so it will make any video, real or not, much less relevant. The tale of new technology bringing war and mayhem might make sense, but in reality it doesn't. You just wish it would happen, because you might want to have some fiction that says "what if?", but it's just a wish. It's also why I dislike black mirror as a tv show. Distopian science fiction is boring most of the time.
- nawtacawp 9y ago
  What a misplaced comment. Perhaps you would like to add the many ways the video could be disproven. Instead, you chose to ramble on about personal preference in sci-fi entertainment.
atmosx 9y ago
Deeper analysis by whom? Who cares about deeper analysis anymore?
jrockway 9y ago
The end game I see is that people won't trust photos or videos anymore. That politician picking up an escort and using racial slurs? Real video. But he'll say "oh, that's just AI, I would never do that" and now he has the plausible deniability. The end result is more freedom and privacy because people won't trust "hard evidence" anymore. I think this might be a good thing.
ouid 9y ago
You can't update the state of the technology and assume that everything else will remain unchanged...
CamperBob2 9y ago
The propaganda potential will be interesting as well. My favorite recent example so far: https://www.youtube.com/watch?v=LYEY4x0_RDY https://www.youtube.com/watch?v=LYEY4x0_RDY With a few obvious exceptions -- like one particular shot of Ivanka's face that crosses the line from realism all the way past comedy into horror -- some of these replacements are almost good enough to start fooling people. Voice synthesis is approaching the 'solved problem' point as well (e.g., https://www.youtube.com/watch?v=YfU_sWHT8mo https://www.youtube.com/watch?v=YfU_sWHT8mo ). Not there yet, but getting there.
yakitori 9y ago
This is why we all should be skeptical of the media ( traditional and social ) as public skepticism is the only thing preventing mass hysteria. If the public were a tad bit more skeptical and didn't believe everything they read in news or social media, we would have avoided the wars of the past 15 years. But every criticism of traditional media here gets flagged. Which is ridiculous for a "hacker" news site when you think about and the hacker ethos.
alexcnwy 9y ago
Most obvious use-case to my mind is stock (or crypto) price manipulation. I wonder what would happen to the ETH price if Vitalik Buterin suddenly "announced" he has seen the light and is quitting ETH to work on DOGE :)
EGreg 9y ago
As I have said for years, our current systems rely on the inefficiency of an attacker. Progress in technology ruins that illusion. Want to play some heads up poker or chess with someone? How sure are you they aren't getting instructions from a program through some means? Don't have anything to hide? How about an AI to dig up plausible sounding parallel construction stories to convince a jury? And in a few decades we're talking about flirting, humor, writing stories, producing any sort of content. There will be a time when a lot of these false positives will still be accepted, until people just stop trusting EVERYTHING.

by mabbo 9y ago

digi_owl 9y ago

> Want to discredit a politician? Here he is picking up a male escort and using racial slurs. Worked so well in a recent election... > These sorts of things could be used to start wars. Wars have been started on much flimsier pretexts, unless you are talking about getting the other side so irate that they fire first.

beisner 9y ago

I think we’ll probably have to move to a model where any security footage or images that is used in court must be digitally signed at the hardware level, with a verification process similar to how CAs work now. Of course, this will still be tamperable, but tampering would have to be much more premeditated and sophisticated.

omeid2 9y ago
The privacy implications of that would be scary. It seems on par with North Korea's Red Star Watermarking which has been dubbed as "dictator’s wet dream". https://qz.com/583311/north-koreas-dissident-tracking-computer-software-is-a-dictators-wet-dream/ https://qz.com/583311/north-koreas-dissident-tracking-comput...
throwaway2048 9y ago
that is absolutely never going to be workable because you can just film another video being played back.
- nine_k 9y ago
  I can imagine watermarking at the light sensor level. It would need some very special provisions for compression, though.
  - striking 9y ago
    Well, but then you'd have to trust the light sensor manufacturer to provide secure keys. Remember when the NSA hacked into Gemalto and had a chance to steal its SIM card keys? However, that's still interesting; isn't it possible that each light sensor has certain watermark-like characteristics arising from the manufacturing process?
    - c22 9y ago
      Yes, fingerprinting of digital cameras has been a thing for at least a decade [https://www.eurekalert.org/pub_releases/2006-04/bu-bur041806.php https://www.eurekalert.org/pub_releases/2006-04/bu-bur041806...]
  - sametmax 9y ago
    Problem is, this way journalists won't have anonymous sources anymore and you will be able to track any citizen filming power abuses.
- beisner 9y ago
  I mean, there are all kinds of ways to bound uncertainty. You’re right, you could film another camera, but that could be addressed separately. Digital signing would only prove that a video came directly from a particular camera’s sensors. To assert the veracity of a video, you’d then have to prove that particular camera was at the scene, and the light coming into the lens was authentic. Most court footage is incidental, not intentional, so the sort of doctoring you describe is unlikely in a majority of cases. Where it is likely, different measures could be taken to bound uncertainty. Separately, generating false images or videos for libel purposes would become more difficult; if a camera that has never been owned by a person is used to produce a signed video that is pornographic in nature, the authenticity of that video is immediately suspect. It would take a much more sophisticated attack to produce a mathematically convincing fake video.

paul7986 9y ago

And that’s why the use and mandating everyone uses their real and verified identity everywhere on the web is needed. You can still comment and post whatever anonymously but it won’t hold any weight!

ameister14 9y ago
That doesn't really make sense in this context; if someone is pretending to be you using your own face and voice, using real identities is kind of the point, isn't it?
- paul7986 9y ago
  Ummm but if it’s not you the uploader thru a verified ID system (block chain maybe) is held liable immediately. Also by the time such a system is out in place the public will be even more suspect of believing what they see on the net as truth. Such is already happening with those 40 and below in terms of finally waking up to all the fakery on the web. Just saw on FB a video where a girl crosses her eyes and they get stuck. Majority of commenters below 40 notes it’s fake. Above 40 commented otherwise. Either we instate an ID system or the web becomes a joke no one believes. The latter I guess wouldn’t be a bad thing.

everdev 9y ago

While the technique does make fraudulent videos easier to make, I don't see this in and of itself as that fatal to public trust. Faking content to hurt someone has been around forever. We then learned not to trust gossip, not to trust uncorroborated sources, etc. I hope it might actually help people be more skeptical of what they're reading and seeing. And hopefully it will help news media be more skeptical of their sources as well. If that doesn't happen then this will be a disaster.

phinnaeus 9y ago
> We then learned not to trust gossip, not to trust uncorroborated sources, etc. Somewhat ironically, I feel a citation is needed here.
- sverige 9y ago
  #MeToo
- Izkata 9y ago
  Makes me think of the phrase "pics or it didn't happen". Granted it was a lot more common before photoshop-style software was everywhere.
cyberpunk0 9y ago
You overestimate people
sametmax 9y ago
> We then learned not to trust gossip, not to trust uncorroborated sources, etc. If it needed any more proof (than facebook, fox, ads, gov propaganda...) that this never happened, I think the Trump debacle should be final evidence this is fantasy.
jaakl 9y ago
The video by itself will be shortly a medium what noone believes as ground truth, just as a written word or a drawing is now. We should be able to construct better cameras with tamperproof footage. But not sure how, as with realtime editing we cannot trust trivial things like secure timestamping. Secure video blockchain anyone?

SanFranManDan 9y ago

All of this has been possible for the last century. For instance, at least in Russia you just get an actor that looks like the politician, film them doing some bad stuff and blackmail them. Pretty simple. We have had doctored photos/videos for as long as the mediums existed. If anything it will saturate and you wont be able to tell what is real or not. Most people will default to "fake" and have to rely on the trustfulness of the source. Just like now. We have photoshop and there isn't mass chaos.

dctoedt 9y ago
> Most people will default to "fake" and have to rely on the trustfulness of the source. Until faked video of the trusted source him- or herself appears.

chrisper 9y ago

These days you don't even seem to need any evidence. An allegation seems to be enough to get you dethroned.

Sir_Cmpwn 9y ago

How long will it take for this approach to be used to splice people into child pornography? The court of public opinion might sentence the victim to maiming or death.

dredmorbius 9y ago
Such things have been known to happen. Emmet Till. The Dutch de Witt brothers in 1672 are a notorious example. False testimony, witness, and evidence are all things. http://tywkiwdbi.blogspot.com/2010/09/corpses-of-de-witt-brothers.html http://tywkiwdbi.blogspot.com/2010/09/corpses-of-de-witt-bro...

jcfrei 9y ago

The problem I see with tech like this is that it muddies the water for everyone. I find it very strange that technologies like the internet appear to have created a world with more ambiguity and less transparency than we've had before - mostly because people now can find support for any kind of belief online. Some people are either willing to learn or have learned how to get information from reputation based systems and some will just go rogue and create their own reality. I hope democracy survives this shift: Because not only will we have to discuss the facts, now we will also have to discuss: What are the facts?

toAnswerIt 9y ago
I think this is a fallacy. the olden days were way worse. to put in presepective Carrots improving vision is propaganda created by the british yet see how many people believed it. Humans inventing and accepting fake beliefs to comfort themselves was never a new thing. Governments gaslighting its own citizens is communist party 101. People always want to believe that yesterday was better and blame their problems on the new.
- gaius 9y ago
  Carrots improving vision is propaganda created by the british yet see how many people believed it. Not exactly propaganda, more misdirection, maskirovka our Russian comrades call it
vertex-four 9y ago
We've always had to discuss what the facts are - the extreme bias between newspapers and TV channels picking and choosing facts to provide to their users, even on the same news stories - news source A chooses not to provide background info, news source B chooses to provide background info from one primary source only, news source C chooses to provide background info from another source, news source D chooses to provide a general overview but with a slight bent towards assuming that its readers hold values X, Y and Z, etc etc. You can go back and read stories from the 1950s which have these issues and more. There has never been a time where all individuals involved in a democracy have understood all the facts. Popular politics has always been driven by newspapers and special interest groups. In my country, implementation of a certain policy has very clear correlation with a marked increase in suicide rates - the slow rollout of this specific policy across areas of the country provides an easy way to show this. Certain newspapers refuse to report on this - it doesn't match their ideology - after all, when pointed out, people holding this ideology will often point out that suicide is one's own decision and it's not the Government's problem.

jessaustin 9y ago

This just gives more power to "authority". If subjects can't believe their own eyes anymore, who will tell them what to think? NBC would be happy to take the job...

jokoon 9y ago

Well I disagree. There are many ways to disprove such video, and if the technology exists, we already know that it exists, so it will make any video, real or not, much less relevant. The tale of new technology bringing war and mayhem might make sense, but in reality it doesn't. You just wish it would happen, because you might want to have some fiction that says "what if?", but it's just a wish. It's also why I dislike black mirror as a tv show. Distopian science fiction is boring most of the time.

nawtacawp 9y ago
What a misplaced comment. Perhaps you would like to add the many ways the video could be disproven. Instead, you chose to ramble on about personal preference in sci-fi entertainment.

atmosx 9y ago

Deeper analysis by whom? Who cares about deeper analysis anymore?

jrockway 9y ago

The end game I see is that people won't trust photos or videos anymore. That politician picking up an escort and using racial slurs? Real video. But he'll say "oh, that's just AI, I would never do that" and now he has the plausible deniability. The end result is more freedom and privacy because people won't trust "hard evidence" anymore. I think this might be a good thing.

ouid 9y ago

You can't update the state of the technology and assume that everything else will remain unchanged...

CamperBob2 9y ago

The propaganda potential will be interesting as well. My favorite recent example so far: https://www.youtube.com/watch?v=LYEY4x0_RDY https://www.youtube.com/watch?v=LYEY4x0_RDY With a few obvious exceptions -- like one particular shot of Ivanka's face that crosses the line from realism all the way past comedy into horror -- some of these replacements are almost good enough to start fooling people. Voice synthesis is approaching the 'solved problem' point as well (e.g., https://www.youtube.com/watch?v=YfU_sWHT8mo https://www.youtube.com/watch?v=YfU_sWHT8mo ). Not there yet, but getting there.

yakitori 9y ago

This is why we all should be skeptical of the media ( traditional and social ) as public skepticism is the only thing preventing mass hysteria. If the public were a tad bit more skeptical and didn't believe everything they read in news or social media, we would have avoided the wars of the past 15 years. But every criticism of traditional media here gets flagged. Which is ridiculous for a "hacker" news site when you think about and the hacker ethos.

alexcnwy 9y ago

Most obvious use-case to my mind is stock (or crypto) price manipulation. I wonder what would happen to the ETH price if Vitalik Buterin suddenly "announced" he has seen the light and is quitting ETH to work on DOGE :)

EGreg 9y ago

As I have said for years, our current systems rely on the inefficiency of an attacker. Progress in technology ruins that illusion. Want to play some heads up poker or chess with someone? How sure are you they aren't getting instructions from a program through some means? Don't have anything to hide? How about an AI to dig up plausible sounding parallel construction stories to convince a jury? And in a few decades we're talking about flirting, humor, writing stories, producing any sort of content. There will be a time when a lot of these false positives will still be accepted, until people just stop trusting EVERYTHING.