6 ms·

Ethics/Implication question: How much time is left until video/photographic evidence ceases to be a reliable factor in courts? I know that while projects like t

Ethics/Implication question: How much time is left until video/photographic evidence ceases to be a reliable factor in courts? I know that while projects like this are happening, so are projects that work to identify fakes. If the former manages to outpace the latter, this could become a real problem. My mind tends to think of a dystopian extension of cops planting drugs on suspects, where they could literally invent photographic evidence.

deleted 8y ago
[deleted]
aerovistae 8y ago
Would it be possible for phones/cameras to embed cryptographic information in video/photo files in such a manner that proved their origin, and which was tied to the pixels in image/video? I don't know if that's a silly suggestion.
- Donald 8y ago
  Sounds easy to thwart via key extraction from the hardware or simply recording a high-res screen playing the neural-network generated image or video.
  - subcosmos 8y ago
    If a signature was located in the least significant bits of the pixels, you couldn't recover by just filming. If you altered the video, you would need to re-sign.
    - monocasa 8y ago
      I think he's suggesting that you play the video for the camera, and then let the camera sign what it saw.
      - subcosmos 8y ago
        Gotcha I guess in the case of faking political speeches and the like, it becomes some trust model regarding who owns the camera.
- throwawaymath 8y ago
  Sure, you could design a camera that signs photos with a private key stored in a hardware security module, then only trust signed photos and videos. It would have to be resistant to physical tampering and side channel attacks. One way to do this would be to stream bytes to the HSM as they're recorded, which would then output the bytes and their signature for local storage. The tricky bit here is closing the analog loophole (using this camera to record a carefully constructed, high resolution fake) and preventing the HSM from signing anything which wasn't recorded by the camera lens.
  - drdeca 8y ago
    If GPS satellites were to cryptographically sign their signals, would that be sufficient to prevent spoofing of GPS signals (assuming the receiver of the signals treated the signing appropriately)? Like, I can imagine maybe there could be an attack where one could record gps signals at some nearby place, and then play them back in slightly different orders/rates to try to fool a receiving device into thinking it is somewhere else (nearby). But I don't know how much of a time delay would be needed in order to do that. Could a timestamping service do timestamping quickly enough to prevent this attack for internet connected devices? I tried looking at how the signals for GPS work (like, how frequently it sends time information and how detailed it is for civilians) but it seemed complicated and I got confused and/or didn't try hard enough, so I didn't arrive at an answer for how long it would take to spoof positions if one could only delay real signals that one received. Edit: the purpose of making the gps coords unspoofable would be to make it so that even if the screen-in-front-of-camera attack was done, it would have to be done at the same location and time as it would be claimed to have happened.
    - darkmighty 8y ago
      > If GPS satellites were to cryptographically sign their signals, would that be sufficient to prevent spoofing of GPS signals (assuming the receiver of the signals treated the signing appropriately)? Nope, classic replay attack case (just record the GPS signal and replay to the device at the desired location). You'd need a true time signal within the device, e.g. and atomic clock to make it work (so you'd authenticate the signed time against true time). --- There is another way, however. If we assume the hardware is tamper-proof (unless drastically different methods are needed), then with strict timings we can device a challenge-response system that's immune to replay attacks due to relativity: simply transmit a signal A, have a known 3rd party (e.g. US government servers in cellphone towers) sign your signal Sig(A) and retransmit, and check that the delay matches the propagation delay you'd expect from the cellphone tower distance, plus the fixed (and immutable since it would be gov-controlled) processing delay. Your tamper-proof crypto-camera would record its location and whether it trusts the location. Using cellular signals is also better because GPS doesn't work indoors and is sensitive to interference. Since we're adding a cellular connection to our device, it would also be a good idea to log its position on the state-controlled servers (again can be done with cryptographic safety assuming non-tampered device), along with some kind of intrusion detection system. As soon as it'd detect an attempt at tampering, it would relay this attempt to the servers, storing the intrusion and invalidating the authenticity subsequent recordings; a self-destruction attempt of the key would probably be also wise. --- And now that I think about it, you'd probably want to put several keys/auths in the device, from different organizations -- not only governments. That way if the government authentication is positive but NGO's mismatch, you can suspect a government-backed forgery attempt (analogously vice-versa).
- ci5er 8y ago
  I don't know that I agree with all of their architectural choices, but this company has developed a first-crack at that very thing: https://www.snapshotdna.com/ https://www.snapshotdna.com/
wilde 8y ago
It wasn’t reliable to begin with: https://www.aclu.org/other/whats-wrong-public-video-surveillance https://www.aclu.org/other/whats-wrong-public-video-surveill...
pixelHD 8y ago
With deepfakes this was being discussed. When a group of redditors managed to create a network architecture to put celebs in porn with surprisingly decent results (in some cases), I think a sponsored research group can get better results. - https://www.brookings.edu/blog/order-from-chaos/2018/05/25/the-west-is-ill-prepared-for-the-wave-of-deep-fakes-that-artificial-intelligence-could-unleash/ https://www.brookings.edu/blog/order-from-chaos/2018/05/25/t... - https://www.msnbc.com/hallie-jackson/watch/fake-obama-warning-about-deep-fakes-goes-viral-1214598723984 https://www.msnbc.com/hallie-jackson/watch/fake-obama-warnin... - https://techcrunch.com/2018/06/04/forget-deepfakes-deep-video-portraits-are-way-better-and-worse/ https://techcrunch.com/2018/06/04/forget-deepfakes-deep-vide...
cookingrobot 8y ago
Even fakeable evidence can be useful. If a hair is found at a crime scene today, you have to take into account the fact that it's possible (if unlikely) that someone could have planted it there. We'll probably end up in a similar situation when you pull video from a security camera, and you'll have to weigh the possibilities of it being real or fake, even if you can't tell either way.
- aalleavitch 8y ago
  Right: falsifying evidence leaves its own trails, and the likelihood of something being falsisfied can be investigated.
scottlegrand2 8y ago
Still some time yet, try recovering young Geoffrey Hinton from the photo from 2011 or so. However, once we can only train for and tweak such knobs, it's going to get real IMO.
Cyphase 8y ago
"ceases to be a reliable factor in courts" and "dystopian extension of cops planting drugs on suspects" seem to be mutually exclusive. Also, as I commented on a HN thread 6 months ago[0], let's remember that there was no 'objective' record of events for the vast majority of human history. It's only in the last 150-200 years that we've had light and sound recording technology. [0] https://news.ycombinator.com/item?id=16014047 https://news.ycombinator.com/item?id=16014047 [Google claims near-human accuracy at imitating a person speaking from text]

deleted 8y ago

[deleted]

aerovistae 8y ago

Would it be possible for phones/cameras to embed cryptographic information in video/photo files in such a manner that proved their origin, and which was tied to the pixels in image/video? I don't know if that's a silly suggestion.

Donald 8y ago
Sounds easy to thwart via key extraction from the hardware or simply recording a high-res screen playing the neural-network generated image or video.
- subcosmos 8y ago
  If a signature was located in the least significant bits of the pixels, you couldn't recover by just filming. If you altered the video, you would need to re-sign.
  - monocasa 8y ago
    I think he's suggesting that you play the video for the camera, and then let the camera sign what it saw.
    - subcosmos 8y ago
      Gotcha I guess in the case of faking political speeches and the like, it becomes some trust model regarding who owns the camera.
throwawaymath 8y ago
Sure, you could design a camera that signs photos with a private key stored in a hardware security module, then only trust signed photos and videos. It would have to be resistant to physical tampering and side channel attacks. One way to do this would be to stream bytes to the HSM as they're recorded, which would then output the bytes and their signature for local storage. The tricky bit here is closing the analog loophole (using this camera to record a carefully constructed, high resolution fake) and preventing the HSM from signing anything which wasn't recorded by the camera lens.
- drdeca 8y ago
  If GPS satellites were to cryptographically sign their signals, would that be sufficient to prevent spoofing of GPS signals (assuming the receiver of the signals treated the signing appropriately)? Like, I can imagine maybe there could be an attack where one could record gps signals at some nearby place, and then play them back in slightly different orders/rates to try to fool a receiving device into thinking it is somewhere else (nearby). But I don't know how much of a time delay would be needed in order to do that. Could a timestamping service do timestamping quickly enough to prevent this attack for internet connected devices? I tried looking at how the signals for GPS work (like, how frequently it sends time information and how detailed it is for civilians) but it seemed complicated and I got confused and/or didn't try hard enough, so I didn't arrive at an answer for how long it would take to spoof positions if one could only delay real signals that one received. Edit: the purpose of making the gps coords unspoofable would be to make it so that even if the screen-in-front-of-camera attack was done, it would have to be done at the same location and time as it would be claimed to have happened.
  - darkmighty 8y ago
    > If GPS satellites were to cryptographically sign their signals, would that be sufficient to prevent spoofing of GPS signals (assuming the receiver of the signals treated the signing appropriately)? Nope, classic replay attack case (just record the GPS signal and replay to the device at the desired location). You'd need a true time signal within the device, e.g. and atomic clock to make it work (so you'd authenticate the signed time against true time). --- There is another way, however. If we assume the hardware is tamper-proof (unless drastically different methods are needed), then with strict timings we can device a challenge-response system that's immune to replay attacks due to relativity: simply transmit a signal A, have a known 3rd party (e.g. US government servers in cellphone towers) sign your signal Sig(A) and retransmit, and check that the delay matches the propagation delay you'd expect from the cellphone tower distance, plus the fixed (and immutable since it would be gov-controlled) processing delay. Your tamper-proof crypto-camera would record its location and whether it trusts the location. Using cellular signals is also better because GPS doesn't work indoors and is sensitive to interference. Since we're adding a cellular connection to our device, it would also be a good idea to log its position on the state-controlled servers (again can be done with cryptographic safety assuming non-tampered device), along with some kind of intrusion detection system. As soon as it'd detect an attempt at tampering, it would relay this attempt to the servers, storing the intrusion and invalidating the authenticity subsequent recordings; a self-destruction attempt of the key would probably be also wise. --- And now that I think about it, you'd probably want to put several keys/auths in the device, from different organizations -- not only governments. That way if the government authentication is positive but NGO's mismatch, you can suspect a government-backed forgery attempt (analogously vice-versa).
ci5er 8y ago
I don't know that I agree with all of their architectural choices, but this company has developed a first-crack at that very thing: https://www.snapshotdna.com/ https://www.snapshotdna.com/

wilde 8y ago

It wasn’t reliable to begin with: https://www.aclu.org/other/whats-wrong-public-video-surveillance https://www.aclu.org/other/whats-wrong-public-video-surveill...

pixelHD 8y ago

With deepfakes this was being discussed. When a group of redditors managed to create a network architecture to put celebs in porn with surprisingly decent results (in some cases), I think a sponsored research group can get better results. - https://www.brookings.edu/blog/order-from-chaos/2018/05/25/the-west-is-ill-prepared-for-the-wave-of-deep-fakes-that-artificial-intelligence-could-unleash/ https://www.brookings.edu/blog/order-from-chaos/2018/05/25/t... - https://www.msnbc.com/hallie-jackson/watch/fake-obama-warning-about-deep-fakes-goes-viral-1214598723984 https://www.msnbc.com/hallie-jackson/watch/fake-obama-warnin... - https://techcrunch.com/2018/06/04/forget-deepfakes-deep-video-portraits-are-way-better-and-worse/ https://techcrunch.com/2018/06/04/forget-deepfakes-deep-vide...

cookingrobot 8y ago

Even fakeable evidence can be useful. If a hair is found at a crime scene today, you have to take into account the fact that it's possible (if unlikely) that someone could have planted it there. We'll probably end up in a similar situation when you pull video from a security camera, and you'll have to weigh the possibilities of it being real or fake, even if you can't tell either way.

aalleavitch 8y ago
Right: falsifying evidence leaves its own trails, and the likelihood of something being falsisfied can be investigated.

scottlegrand2 8y ago

Still some time yet, try recovering young Geoffrey Hinton from the photo from 2011 or so. However, once we can only train for and tweak such knobs, it's going to get real IMO.

Cyphase 8y ago

"ceases to be a reliable factor in courts" and "dystopian extension of cops planting drugs on suspects" seem to be mutually exclusive. Also, as I commented on a HN thread 6 months ago[0], let's remember that there was no 'objective' record of events for the vast majority of human history. It's only in the last 150-200 years that we've had light and sound recording technology. [0] https://news.ycombinator.com/item?id=16014047 https://news.ycombinator.com/item?id=16014047 [Google claims near-human accuracy at imitating a person speaking from text]