5 ms·

I have a really bad feeling about this technique. Change your webcam output to 3 fps, mumble something about a bad connection, and Bob's your uncle. But more

I have a really bad feeling about this technique. Change your webcam output to 3 fps, mumble something about a bad connection, and Bob's your uncle. But more fundamentally: If I am deepfaking my appearance on video call, I have full control of my machine. I thus can (with negligible lag) just grab the screen content. Once I have that, it's really not too much work to write a filter projecting this screen onto the deepfake image. In other words: as a 'security by obscurity' method this might have worked for a while, with this now being public knowledge, I expect deepfake software to defeat this within the year, as this really is not a computationally or architecturally demanding change. It is ultimately just a filter, corresponding to screen content, mirrored and super-blurred. That shouldn't have any discernible lag with a modern computer.

heyitsguay 4y ago
The model here is about using a challenge window with an illumination sequence specifically designed to change faster than can be incorporated into live deepfake models on the fly without creating telltale artifacts from improper approximation of the illumination effects on the subject. Correctly faking the effects of monitor illumination is a non trivial challenge for well-chosen illumination patterns.
- hansword 4y ago
  I read the article. I was proposing to skip the (computationally expensive) step of approximation of the illumination effects on the subject within the live deepfake model, and just do a (computationally cheap) filter over the webcam image to be sent out. I was saying 'correctly faking' might not be necessary, so let's fake it incorrectly. Of course I haven't tested it, but from reading the article, this seems to be enough to fool their detection model. If I misunderstood this, please correct me, I am eager to learn. But please don't just repeat the article back at me. Thank you.
- wokwokwok 4y ago
  > Correctly faking the effects of monitor illumination is a non trivial challenge for well-chosen illumination patterns. Surely not. If you have a face in a specific orientation and lighting the whole point of deep fakes is that it generates “style transfer” of the incoming image, including lighting. Doing it in real time is a computational restriction that makes it expensive, not difficult. Are you really suggesting that the deepfake models can’t accurately reproduce structured light and shadow patterns? I find that very surprising, and not what the paper says (“ We verified that the deep fakes created by Avatar- ify (github.com/alievk/avatarify-python) do not incorporate the environmental lighting and are therefore easily identifiable because in the presence of our active illu- mination, their temporal facial hue is flatlined with a nearly zero correlation.” <— they didn’t even test it except on a system that doesn’t do lighting transfer). …so, this feels a lot more like “well, if you’re doing a crappy off the shelf meme using deepfakelive it’s pretty obvious” rather than “detect a convincing DF”. The approach probably doesn’t scale against more sophisticated models or opponents with adequate resources.
planetsprite 4y ago
The next 10 years of software dev will be defined by the arms race of fraud generation (deepfakes, GAN images, GPT-X bots) and detector bots. What will end up happening is the detectors will catch the bottom 99% of fraudulent content, leaving the top 1% free reign over the dataspace.
- 14 4y ago
  I personally like that deepfakes exist. Anything can be real and anything can be a lie. Plausible deniability for all.
  - danuker 4y ago
    Indeed. And it forces everyone to think critically. No pictures, audio or video online are trustworthy by themselves. Even in science, you can never be completely sure of anything. "Sure enough" for purposes of human life, perhaps. But you can invalidate anything: https://en.wikipedia.org/wiki/Karl_Popper https://en.wikipedia.org/wiki/Karl_Popper
    - _0ffh 4y ago
      Potentially resulting in even more division and radicalisation. The human tendency is to judge evidence strongly subjected to priors. Does the evidence allow me to keep my beliefs, or does it force me to change them. Evidence of questionable authenticity will be believed if it reinforces the priors, and disbelieved if it contradicts them, thereby increasing the divergence of held beliefs.
      - danuker 4y ago
        If you want to find truth, and you understand the nature of reality and perception, it will make you want to seek out counterexamples to your beliefs. I am thankful for anyone pointing out inconsistencies between my own statements and reality. It will let me learn more easily, and refine my beliefs.
Mordisquitos 4y ago
> If I am deepfaking my appearance on video call, I have full control of my machine. I thus can (with negligible lag) just grab the screen content. Once I have that, it's really not too much work to write a filter projecting this screen onto the deepfake image. The article already addresses this, in its fourth paragraph: > The theory behind the approach is that live deepfake systems cannot respond in time to the changes depicted in the on-screen graphic, increasing the ‘lag’ of the deepfake effect at certain parts of the color spectrum, revealing its presence.
- hansword 4y ago
  I was proposing to skip the (computationally expensive) step of approximation of the illumination effects on the subject within the live deepfake model, and just do a (computationally cheap) filter over the webcam image to be sent out. I was saying 'correctly faking' might not be necessary, so let's fake it incorrectly. Of course I haven't tested it, but from reading the article, this seems to be enough to fool their detection model.

heyitsguay 4y ago

The model here is about using a challenge window with an illumination sequence specifically designed to change faster than can be incorporated into live deepfake models on the fly without creating telltale artifacts from improper approximation of the illumination effects on the subject. Correctly faking the effects of monitor illumination is a non trivial challenge for well-chosen illumination patterns.

hansword 4y ago
I read the article. I was proposing to skip the (computationally expensive) step of approximation of the illumination effects on the subject within the live deepfake model, and just do a (computationally cheap) filter over the webcam image to be sent out. I was saying 'correctly faking' might not be necessary, so let's fake it incorrectly. Of course I haven't tested it, but from reading the article, this seems to be enough to fool their detection model. If I misunderstood this, please correct me, I am eager to learn. But please don't just repeat the article back at me. Thank you.
wokwokwok 4y ago
> Correctly faking the effects of monitor illumination is a non trivial challenge for well-chosen illumination patterns. Surely not. If you have a face in a specific orientation and lighting the whole point of deep fakes is that it generates “style transfer” of the incoming image, including lighting. Doing it in real time is a computational restriction that makes it expensive, not difficult. Are you really suggesting that the deepfake models can’t accurately reproduce structured light and shadow patterns? I find that very surprising, and not what the paper says (“ We verified that the deep fakes created by Avatar- ify (github.com/alievk/avatarify-python) do not incorporate the environmental lighting and are therefore easily identifiable because in the presence of our active illu- mination, their temporal facial hue is flatlined with a nearly zero correlation.” <— they didn’t even test it except on a system that doesn’t do lighting transfer). …so, this feels a lot more like “well, if you’re doing a crappy off the shelf meme using deepfakelive it’s pretty obvious” rather than “detect a convincing DF”. The approach probably doesn’t scale against more sophisticated models or opponents with adequate resources.

planetsprite 4y ago

The next 10 years of software dev will be defined by the arms race of fraud generation (deepfakes, GAN images, GPT-X bots) and detector bots. What will end up happening is the detectors will catch the bottom 99% of fraudulent content, leaving the top 1% free reign over the dataspace.

14 4y ago
I personally like that deepfakes exist. Anything can be real and anything can be a lie. Plausible deniability for all.
- danuker 4y ago
  Indeed. And it forces everyone to think critically. No pictures, audio or video online are trustworthy by themselves. Even in science, you can never be completely sure of anything. "Sure enough" for purposes of human life, perhaps. But you can invalidate anything: https://en.wikipedia.org/wiki/Karl_Popper https://en.wikipedia.org/wiki/Karl_Popper
  - _0ffh 4y ago
    Potentially resulting in even more division and radicalisation. The human tendency is to judge evidence strongly subjected to priors. Does the evidence allow me to keep my beliefs, or does it force me to change them. Evidence of questionable authenticity will be believed if it reinforces the priors, and disbelieved if it contradicts them, thereby increasing the divergence of held beliefs.
    - danuker 4y ago
      If you want to find truth, and you understand the nature of reality and perception, it will make you want to seek out counterexamples to your beliefs. I am thankful for anyone pointing out inconsistencies between my own statements and reality. It will let me learn more easily, and refine my beliefs.

Mordisquitos 4y ago

> If I am deepfaking my appearance on video call, I have full control of my machine. I thus can (with negligible lag) just grab the screen content. Once I have that, it's really not too much work to write a filter projecting this screen onto the deepfake image. The article already addresses this, in its fourth paragraph: > The theory behind the approach is that live deepfake systems cannot respond in time to the changes depicted in the on-screen graphic, increasing the ‘lag’ of the deepfake effect at certain parts of the color spectrum, revealing its presence.

hansword 4y ago
I was proposing to skip the (computationally expensive) step of approximation of the illumination effects on the subject within the live deepfake model, and just do a (computationally cheap) filter over the webcam image to be sent out. I was saying 'correctly faking' might not be necessary, so let's fake it incorrectly. Of course I haven't tested it, but from reading the article, this seems to be enough to fool their detection model.