4 ms·

Long term, the only robust way to solve this is going to involve a remote attestation chain i.e. video that's being signed by the web cam as it's produced, and

Long term, the only robust way to solve this is going to involve a remote attestation chain i.e. video that's being signed by the web cam as it's produced, and then transformed/recompressed inside e.g. SGX enclaves or an SEV protected virtual machine that's sending an RA to the other side. Although hard to set up (you need a lot of people to cooperate and CPU vendors have to bring these features back to consumer hardware), it has a lot of advantages over what you might call trick-based approaches: 1. Robust to AI improvements. 2. Blocks all kinds of faking and tampering, not just deepfakes. 3. With a bit of work can securely timestamp the video such that it can become evidence useful for dispute resolution. 4. Also applies to audio. 5. Works in the static/offline scenario where you just get a video file and have to check it. There are probably other advantages too. The way to do such things has been known about for a long time. The issue is not any missing pieces of tech but simply building a consensus amongst hardware vendors that there's actual market demand for [deep]fake-proof IO. In reality, deepfakes have been around for some years now but have there been any reports of actual real world attacks using them? Not sure, I didn't hear of any but maybe there's been one or two. Problem is, that's not enough to sustain a market. Attacks have to become pretty common before it's worth throwing anything more than cheap heuristics at it.

ballenf 4y ago
Then you just point the webcam at a screen or microphone at a speaker? I really don't think moving our trust to unknown, unnamed manufacturers of hardware in far away places is a solution. The solution is not going to be high tech, imho. Just like we have learned a skepticism resulting from Photoshop, we'll learn a skepticism of live video or audio.
- oliwarner 4y ago
  You could layer on IR depth mapping, available in many Windows Hello providing camera systems. I happen to agree with the other voices here saying this is a folly game of cat and mouse, but there are near-time methods of making this harder to fool. And that might be enough for now.
- itake 4y ago
  The typical trick is to flash colors on the screen facing the user. The light bounces back off of skin differently than a screen.
Starlevel001 4y ago
Applying technological solutions to social problems hasn't worked a single time before, but SURELY it'll work this time
- shockeychap 4y ago
  Encryption and use of signed certificates has certainly been a big help against web fraud. No, it's not perfect, and can't prevent certain kinds of phishing, but it has definitely raised the bar for would-be scammers. It makes it nearly impossible to spoof "amazon.com" in the browser, and it prevents passive snooping on open WiFi. You can't make it impossible, but you can make it very difficult. My elderly uncle almost gave $10,000 to a scammer who had convinced him that his nephew was sitting in a jail and needed this money to be paid for his bail. Luckily, he reached out to me for help and I was able to confirm that his nephew was at home, not in jail. I honestly can't imagine some of the scams that are coming, particularly to the tech-vulnerable, if we don't do SOMETHING to make real-time deepfake video harder than it now is.
- mike_hearn 4y ago
  DeepFakes are a technological problem, not a social problem.
jedberg 4y ago
The only solution will be in person meetings, as it has always been. Faking audio has been around a really long time. If you needed to be absolutely sure the person you're talking to is legit, you met them in person (mission impossible style disguises not withstanding). Nothing has really changed with deepfake, other than the fact that for a brief period we could be sure the person we were having a video chat with was legit because the tech didn't exist to fake it.
xen2xen1 4y ago
So your answer is .. more DRM?
feanaro 4y ago
The solution you propose sounds vastly overengineered. Why would we need remote attestation, tampering resistance and enclaves when this is simply a problem of your peers being unauthenticated? If you care about the identity of who you are speaking to remotely, the only solution is to cryptographically verify the other end, which just requires plain old key distribution and verification. It's just not widespread enough today for videocalls because up to now, there wasn't much need for this.
- judge2020 4y ago
  How do you verify they are who they say they are, though? And verifying their picture matches their name?
  - feanaro 4y ago
    What are the practical scenarios where you need this that would require the introduction of overengineered, Orwellian, crippled computers described above, while at the same time not being solvable by other more realistic means?
  - chii 4y ago
    you don't. You assume that the key distribution mechanism is secure, so that whoever has the key _is_ who they say they are. This doesn't prevent adversarial impersonation (where you cannot trust the party that want themselves impersonated). E.g., if you are an employer, and interviewing via video, you cannot tell that the person authenticated and performing on video is indeed the person you're hiring. I dont think this is a problem that _should_ be solved tbh.
    - mike_hearn 4y ago
      That just moves the problem around, it doesn't solve it. It's also incompatible with anti-money laundering laws that don't make it easy/safe to outsource ID verification.
      - 4y ago
IMSAI8080 4y ago
I think it would be useful if news outlets signed their video content using watermarking techniques. Then social media sites where news is shared could automatically check for recognised signatures for major outlets and give it a checkmark or something. The signature could be easily removed but video without the checkmark would then be suspicious. It would also be useful if they added signed timecodes to frames so it could be checked if the video has been edited.

ballenf 4y ago

Then you just point the webcam at a screen or microphone at a speaker? I really don't think moving our trust to unknown, unnamed manufacturers of hardware in far away places is a solution. The solution is not going to be high tech, imho. Just like we have learned a skepticism resulting from Photoshop, we'll learn a skepticism of live video or audio.

oliwarner 4y ago
You could layer on IR depth mapping, available in many Windows Hello providing camera systems. I happen to agree with the other voices here saying this is a folly game of cat and mouse, but there are near-time methods of making this harder to fool. And that might be enough for now.
itake 4y ago
The typical trick is to flash colors on the screen facing the user. The light bounces back off of skin differently than a screen.

Starlevel001 4y ago

Applying technological solutions to social problems hasn't worked a single time before, but SURELY it'll work this time

shockeychap 4y ago
Encryption and use of signed certificates has certainly been a big help against web fraud. No, it's not perfect, and can't prevent certain kinds of phishing, but it has definitely raised the bar for would-be scammers. It makes it nearly impossible to spoof "amazon.com" in the browser, and it prevents passive snooping on open WiFi. You can't make it impossible, but you can make it very difficult. My elderly uncle almost gave $10,000 to a scammer who had convinced him that his nephew was sitting in a jail and needed this money to be paid for his bail. Luckily, he reached out to me for help and I was able to confirm that his nephew was at home, not in jail. I honestly can't imagine some of the scams that are coming, particularly to the tech-vulnerable, if we don't do SOMETHING to make real-time deepfake video harder than it now is.
mike_hearn 4y ago
DeepFakes are a technological problem, not a social problem.

jedberg 4y ago

The only solution will be in person meetings, as it has always been. Faking audio has been around a really long time. If you needed to be absolutely sure the person you're talking to is legit, you met them in person (mission impossible style disguises not withstanding). Nothing has really changed with deepfake, other than the fact that for a brief period we could be sure the person we were having a video chat with was legit because the tech didn't exist to fake it.

xen2xen1 4y ago

So your answer is .. more DRM?

feanaro 4y ago

The solution you propose sounds vastly overengineered. Why would we need remote attestation, tampering resistance and enclaves when this is simply a problem of your peers being unauthenticated? If you care about the identity of who you are speaking to remotely, the only solution is to cryptographically verify the other end, which just requires plain old key distribution and verification. It's just not widespread enough today for videocalls because up to now, there wasn't much need for this.

judge2020 4y ago
How do you verify they are who they say they are, though? And verifying their picture matches their name?
- feanaro 4y ago
  What are the practical scenarios where you need this that would require the introduction of overengineered, Orwellian, crippled computers described above, while at the same time not being solvable by other more realistic means?
- chii 4y ago
  you don't. You assume that the key distribution mechanism is secure, so that whoever has the key _is_ who they say they are. This doesn't prevent adversarial impersonation (where you cannot trust the party that want themselves impersonated). E.g., if you are an employer, and interviewing via video, you cannot tell that the person authenticated and performing on video is indeed the person you're hiring. I dont think this is a problem that _should_ be solved tbh.
  - mike_hearn 4y ago
    That just moves the problem around, it doesn't solve it. It's also incompatible with anti-money laundering laws that don't make it easy/safe to outsource ID verification.
    - 4y ago

IMSAI8080 4y ago

I think it would be useful if news outlets signed their video content using watermarking techniques. Then social media sites where news is shared could automatically check for recognised signatures for major outlets and give it a checkmark or something. The signature could be easily removed but video without the checkmark would then be suspicious. It would also be useful if they added signed timecodes to frames so it could be checked if the video has been edited.