5 ms·

I can't imagine this kind of analysis carrying much weight in 3-5 years as advances in machine learning and image manipulation turn the notion of "seeing is bel

by apo 8y ago

I can't imagine this kind of analysis carrying much weight in 3-5 years as advances in machine learning and image manipulation turn the notion of "seeing is believing" on its head. edit: Imagine running this process in reverse. Pick the culprit, time, and location. Then arrange the scene to exactly match the false accusation. Probably didn't happen here, but in a few years it won't be possible to know the difference.

browsercoin 8y ago
I disagree that "deepfakes" will be unverifiable. There is very robust forensic framework for analyzing videos for authenticity, and it often relies not just on the visual component but all the other vectors that open up as a result-EM waves being generated, audio analysis, etc. A deepfake that passes this analytical function would imply that we will not be seeing advancements in digital forensics due to AI, which does not make sense because there is no reason not to update forensic processes to keep up with changing technology.
- wish5031 8y ago
  That seems like a plausible argument, but there’s a huge portion of the population that’ll willingly believe any realistic-looking video they see, regardless of the evidence. That could have fatal consequences, e.g. as happened with the Planned Parenthood video in 2015. [0] [0] https://en.m.wikipedia.org/wiki/Planned_Parenthood_2015_undercover_videos_controversy https://en.m.wikipedia.org/wiki/Planned_Parenthood_2015_unde...
  - fossuser 8y ago
    It doesn't even have to be realistic, people will believe outrageous claims or lies that conflict current video evidence because it fits a narrative they want to be true. It seems that trying to actually determine what's true as opposed to just attempting to frame issues in support of your team (or what you already currently believe to be true) is a relatively rare quality. I guess the silver lining is that while high quality fake videos may make things worse, there's already a pretty deep divide between people who will try and figure out what's true and people who don't. Higher quality fakes may only grab a small percentage of additional people.
  - Consultant32452 8y ago
    I agree. People supported a war in Iraq based largely on cartoonish photoshops of mobile nuclear weapons labs in ice cream trucks.
- ezoe 8y ago
  If there is a program which output the authenticity of input, Can't we use that to further improve the NN? Eventually, given enough computation, trained NN reliably output videos even the said program can't bust. I see a dark future ahead of us.
  - acjohnson55 8y ago
    No darker than the past. Although, to be fair, the past was pretty dark.
  - oh_sigh 8y ago
    This is the ideas behind GANs and progress is reliant upon the generator and discriminator generally can't get too far ahead of one another
psergeant 8y ago
I submitted basically this: https://github.com/pjlsergeant/multimedia-trust-and-certification https://github.com/pjlsergeant/multimedia-trust-and-certific... to YC this year. Definitely something that needs thinking about
- nathan_f77 8y ago
  That's awesome! Haha I also have a repo like this, where I was putting together some research around identity and zero-knowledge proofs: https://github.com/ndbroadbent/sessionproof https://github.com/ndbroadbent/sessionproof Zero-knowledge proofs might also have some applications for multimedia verification.
synthmeat 8y ago
It's not that it will be impossible to detect (they probably will be for quite some time) as it's about nature of dissemination - you first see fake, and only in its echoes you possibly see the truth. Now, problem is, if fakes are easy to make, there'll be more of them, particulary if they generate impressions and spread faster than rebutals. And as space of fakes is infinite compared to space of truth, with advanced tools, it can be concocted in such a way that falsifying it might be lengthy process. And, in some cases there, it may be too late.
- deleted 8y ago
  [deleted]
kelp 8y ago
I keep meaning to write up a longer form version of this idea that I've been kicking around to solve this. This issue is often talked about as an unsolvable problem, but I feel like we already have the technology to deal with it. Lots of devices, such as an iPhone already have a secure enclave that can be used for identity. Why not use this to sign videos and images for authentication purposes? Then certain devices can produce verified, undoctored images and videos. Or even allow some amount of limited tweaking, and editing while attesting to the original. This could be embedded into the file produced, like exif data, and read by services like Facebook and Twitter to badge the image/video as verified and undoctored. This could improve the amature case significantly. Professional photography and video equipment could have similar capabilities so originals could be produced and proven to be genuine. I guess this idea still depends on the hardware devices remaining secure, which is far from trivial, but it does increase the cost of producing a fake. And to keep producing really good fakes, you have to have a zero day to fake a genuine signature, or it becomes know that any media produced with that device version is suspect. If such a system became widly adopted, at the very least, savvy users, and the media would be suspect of anything not signed. And perhaps the general public would also learn to be skeptical, particularly if the issue of fakes became more widespread.
- CraneWorm 8y ago
  I think it remains unsolvable: you cannot plug the analog hole. Hardware device can be as secure as you like, but you can record your fake output to sign it and it suddenly becomes very genuine, I don't think it bumps the cost sufficiently.
- nathan_f77 8y ago
  Haha I was also thinking about using an HSM module to sign videos [1]. I think the problem is that anyone can just record a projector screen and replay audio through some speakers, and then Facebook and Twitter will still badge the video as verified and undoctored. You could detect this by analyzing the video and audio, but then it's just an arms race with machine learning. [1] https://news.ycombinator.com/item?id=18074425 https://news.ycombinator.com/item?id=18074425
- PeterisP 8y ago
hbosch 8y ago
> Pick the culprit, time, and location. Then arrange the scene to exactly match the false accusation. Ah yes, but I logged on to Facebook at 8:12am and scrolled my news feed for a moment when I clicked a Chevrolet ad at exactly 8:15am (Facebook knows). I watched the video for exactly 12 seconds (Facebook still knows), then texted to my wife at 8:16am that we should think about getting a new car. I open up Reddit and log in at 8:17am and comment in /r/cars at 8:25 (Reddit knows). I started my Tesla at 8:32am and drove for 28 minutes (Tesla knows) to a Whole Foods, where I saved the location of my parked car (at almost exactly 9:00am) and entered the store. Grabbed a cold brew in a can and paid with my Visa card at 9:04 am (Visa knows), then walked to my office building and badged in through the door at 9:18 (my company knows), just in time to log on to the corporate network and send off a couple e-mails at 9:25, 9:26, and 9:29am. My whole day is tracked, timestamped and corroborated by third-parties already! If we were able to easily collate the data that our workplaces, our GPS, our phones have on us it would be virtually impossible to place someone in a place they are not! And to go full dystopian, it's not too big a stretch to imagine the government would love to have exactly this.
nathan_f77 8y ago
I think you're probably right. I wonder if smartphone makers could include some kind of hardware chip to verify the authenticity of videos. I don't think it's possible to build an unbreakable solution, but you could just make it very expensive. The camera device could use a chip that included an HSM module, and it could use steganography to create verifiable videos. Each private key could be tied to a unique device ID in the maker's database. This might even be a legitimate use for blockchain. The device maker could create chainpoint proofs that include device IDs and public keys and embed them in Bitcoin transactions. The HSM would be tamper-resistant, so it would be very expensive to extract the private key from the chip, or you'd have to collude with the device maker. All of this is a moot point if you can just jailbreak the phone, fake the GPS and accelerometer data, put the camera in front of a screen, and play some audio into the microphones. I wonder if there are ways to detect video and audio that have been replayed through a projector/speaker instead of recorded in the real world. It's probably difficult to get it pixel perfect and replay perfect audio, and a neural net could probably be trained to spot any errors. Or maybe the projector/audio chip makers could include some steganography in their output. They might include an undetectable signal to indicate that this data is being replayed through a device, instead of recorded in the real world. But then if someone wanted to commit a crime and get away with it, they could reverse engineer the signals from these projector/audio chips, and play those in the background. So any recorded videos could be discounted as a fake. Also, I was thinking about how this could be used to prevent piracy when people record the screens in a movie theater. The studios could include a signal in the video, and the smartphone/video camera makers could stop recording when they detect that signal. But then a criminal could just hold up a screen with some copyrighted content, and everyone's phones and security cameras would stop recording. If phones ever come with light-field cameras, it would also be a lot harder to fake. We'd have to invent realistic light-field projectors, but that would be pretty awesome. I've always wanted a holodeck. This has been fun to think about. Would make some interesting Black Mirror episodes.
tim333 8y ago
Even if they get the image manipulation perfect there will be lots of other ways to double check. Like if the featured video was fake there'd quite likely be evidence that some of the people were elsewhere at the time, or victims still alive and so forth.

by apo 8y ago

browsercoin 8y ago

I disagree that "deepfakes" will be unverifiable. There is very robust forensic framework for analyzing videos for authenticity, and it often relies not just on the visual component but all the other vectors that open up as a result-EM waves being generated, audio analysis, etc. A deepfake that passes this analytical function would imply that we will not be seeing advancements in digital forensics due to AI, which does not make sense because there is no reason not to update forensic processes to keep up with changing technology.

wish5031 8y ago
That seems like a plausible argument, but there’s a huge portion of the population that’ll willingly believe any realistic-looking video they see, regardless of the evidence. That could have fatal consequences, e.g. as happened with the Planned Parenthood video in 2015. [0] [0] https://en.m.wikipedia.org/wiki/Planned_Parenthood_2015_undercover_videos_controversy https://en.m.wikipedia.org/wiki/Planned_Parenthood_2015_unde...
- fossuser 8y ago
  It doesn't even have to be realistic, people will believe outrageous claims or lies that conflict current video evidence because it fits a narrative they want to be true. It seems that trying to actually determine what's true as opposed to just attempting to frame issues in support of your team (or what you already currently believe to be true) is a relatively rare quality. I guess the silver lining is that while high quality fake videos may make things worse, there's already a pretty deep divide between people who will try and figure out what's true and people who don't. Higher quality fakes may only grab a small percentage of additional people.
- Consultant32452 8y ago
  I agree. People supported a war in Iraq based largely on cartoonish photoshops of mobile nuclear weapons labs in ice cream trucks.
ezoe 8y ago
If there is a program which output the authenticity of input, Can't we use that to further improve the NN? Eventually, given enough computation, trained NN reliably output videos even the said program can't bust. I see a dark future ahead of us.
- acjohnson55 8y ago
  No darker than the past. Although, to be fair, the past was pretty dark.
- oh_sigh 8y ago
  This is the ideas behind GANs and progress is reliant upon the generator and discriminator generally can't get too far ahead of one another

psergeant 8y ago

I submitted basically this: https://github.com/pjlsergeant/multimedia-trust-and-certification https://github.com/pjlsergeant/multimedia-trust-and-certific... to YC this year. Definitely something that needs thinking about

nathan_f77 8y ago
That's awesome! Haha I also have a repo like this, where I was putting together some research around identity and zero-knowledge proofs: https://github.com/ndbroadbent/sessionproof https://github.com/ndbroadbent/sessionproof Zero-knowledge proofs might also have some applications for multimedia verification.

synthmeat 8y ago

It's not that it will be impossible to detect (they probably will be for quite some time) as it's about nature of dissemination - you first see fake, and only in its echoes you possibly see the truth. Now, problem is, if fakes are easy to make, there'll be more of them, particulary if they generate impressions and spread faster than rebutals. And as space of fakes is infinite compared to space of truth, with advanced tools, it can be concocted in such a way that falsifying it might be lengthy process. And, in some cases there, it may be too late.

deleted 8y ago
[deleted]

kelp 8y ago

I keep meaning to write up a longer form version of this idea that I've been kicking around to solve this. This issue is often talked about as an unsolvable problem, but I feel like we already have the technology to deal with it. Lots of devices, such as an iPhone already have a secure enclave that can be used for identity. Why not use this to sign videos and images for authentication purposes? Then certain devices can produce verified, undoctored images and videos. Or even allow some amount of limited tweaking, and editing while attesting to the original. This could be embedded into the file produced, like exif data, and read by services like Facebook and Twitter to badge the image/video as verified and undoctored. This could improve the amature case significantly. Professional photography and video equipment could have similar capabilities so originals could be produced and proven to be genuine. I guess this idea still depends on the hardware devices remaining secure, which is far from trivial, but it does increase the cost of producing a fake. And to keep producing really good fakes, you have to have a zero day to fake a genuine signature, or it becomes know that any media produced with that device version is suspect. If such a system became widly adopted, at the very least, savvy users, and the media would be suspect of anything not signed. And perhaps the general public would also learn to be skeptical, particularly if the issue of fakes became more widespread.

CraneWorm 8y ago
I think it remains unsolvable: you cannot plug the analog hole. Hardware device can be as secure as you like, but you can record your fake output to sign it and it suddenly becomes very genuine, I don't think it bumps the cost sufficiently.
nathan_f77 8y ago
Haha I was also thinking about using an HSM module to sign videos [1]. I think the problem is that anyone can just record a projector screen and replay audio through some speakers, and then Facebook and Twitter will still badge the video as verified and undoctored. You could detect this by analyzing the video and audio, but then it's just an arms race with machine learning. [1] https://news.ycombinator.com/item?id=18074425 https://news.ycombinator.com/item?id=18074425
PeterisP 8y ago

hbosch 8y ago

> Pick the culprit, time, and location. Then arrange the scene to exactly match the false accusation. Ah yes, but I logged on to Facebook at 8:12am and scrolled my news feed for a moment when I clicked a Chevrolet ad at exactly 8:15am (Facebook knows). I watched the video for exactly 12 seconds (Facebook still knows), then texted to my wife at 8:16am that we should think about getting a new car. I open up Reddit and log in at 8:17am and comment in /r/cars at 8:25 (Reddit knows). I started my Tesla at 8:32am and drove for 28 minutes (Tesla knows) to a Whole Foods, where I saved the location of my parked car (at almost exactly 9:00am) and entered the store. Grabbed a cold brew in a can and paid with my Visa card at 9:04 am (Visa knows), then walked to my office building and badged in through the door at 9:18 (my company knows), just in time to log on to the corporate network and send off a couple e-mails at 9:25, 9:26, and 9:29am. My whole day is tracked, timestamped and corroborated by third-parties already! If we were able to easily collate the data that our workplaces, our GPS, our phones have on us it would be virtually impossible to place someone in a place they are not! And to go full dystopian, it's not too big a stretch to imagine the government would love to have exactly this.

nathan_f77 8y ago

I think you're probably right. I wonder if smartphone makers could include some kind of hardware chip to verify the authenticity of videos. I don't think it's possible to build an unbreakable solution, but you could just make it very expensive. The camera device could use a chip that included an HSM module, and it could use steganography to create verifiable videos. Each private key could be tied to a unique device ID in the maker's database. This might even be a legitimate use for blockchain. The device maker could create chainpoint proofs that include device IDs and public keys and embed them in Bitcoin transactions. The HSM would be tamper-resistant, so it would be very expensive to extract the private key from the chip, or you'd have to collude with the device maker. All of this is a moot point if you can just jailbreak the phone, fake the GPS and accelerometer data, put the camera in front of a screen, and play some audio into the microphones. I wonder if there are ways to detect video and audio that have been replayed through a projector/speaker instead of recorded in the real world. It's probably difficult to get it pixel perfect and replay perfect audio, and a neural net could probably be trained to spot any errors. Or maybe the projector/audio chip makers could include some steganography in their output. They might include an undetectable signal to indicate that this data is being replayed through a device, instead of recorded in the real world. But then if someone wanted to commit a crime and get away with it, they could reverse engineer the signals from these projector/audio chips, and play those in the background. So any recorded videos could be discounted as a fake. Also, I was thinking about how this could be used to prevent piracy when people record the screens in a movie theater. The studios could include a signal in the video, and the smartphone/video camera makers could stop recording when they detect that signal. But then a criminal could just hold up a screen with some copyrighted content, and everyone's phones and security cameras would stop recording. If phones ever come with light-field cameras, it would also be a lot harder to fake. We'd have to invent realistic light-field projectors, but that would be pretty awesome. I've always wanted a holodeck. This has been fun to think about. Would make some interesting Black Mirror episodes.

tim333 8y ago

Even if they get the image manipulation perfect there will be lots of other ways to double check. Like if the featured video was fake there'd quite likely be evidence that some of the people were elsewhere at the time, or victims still alive and so forth.