4 ms·

A few weeks ago, there was a post here about that guy who deepfaked female profile pictures. He showed that the visible artifacts go away by merely scaling up t

A few weeks ago, there was a post here about that guy who deepfaked female profile pictures. He showed that the visible artifacts go away by merely scaling up the number of parameters in the model. Give it a few years and these fake images will be impossible to detect unless we agree on some way of cryptographically signing images from real cameras.

sideshowb 5y ago
I'm not hopeful that the latter will do much. Just do your deepfake, point a camera at your screen and press the shutter button to get it cryptographically signed..? Ok, so that's tongue in cheek and we'd see some artifacts there but the general principle works. You could intercept the signal from the CCD, or just extract the signing key from the camera's ROM, etc etc.
- sigmoid10 5y ago
  You can already do that today to fool basic image forensics. Security is never absolute, but if you make the hassle big enough it might suffice to protect the general population. Just like almost all smartphones don't let you mess with the wifi card or spoof your mac address unless you root them. If that wasn't the case, you'd see way more hostapd-wpe attacks these days, since every kid could do it.
  - rspeele 5y ago
    In the case of image trustworthiness, it might actually be better for it to be accessible to the general population, if it's doable at all. Then the public knows that images are unreliable evidence.
foobar33333 5y ago
Cryptographic keys embedded in hardware worked really well for the DRM industry. Eventually someone will work out how to extract the key, then create malware to spread around android devices to send back millions of valid keys. What do you do then? Blacklist millions of real peoples cameras and prevent them from using the internet?
- sigmoid10 5y ago
  You only need a method of key invalidation and renewal, that's all. Ssl certificates have been facing this problem for years. The threat model is equal to someone infecting millions of devices and then sending back banking data, so it's not like people aren't working on mitigating that stuff.
  - sebzim4500 5y ago
    >Ssl certificates have been facing this problem for years. That's an entirely different, and much easier problem. In the case of SSL you are not worried about an attacker obtaining a certificate for any host, just the ones you care about. E.g. an attacker getting a certificate for facebook.com would be catastrophic, but an attacker getting a certificate for a website that no one uses would be a non-issue. For the case of avoiding deepfakes, you need to avoid the attacker extracting a key from any of the millions of cameras that are sold every year.
    - uuidgen 5y ago
      Every camera will have a unique key so a single key leaked is still no issue. It's easy to maintain a blacklist - similarly how CRLs work.
      - xrendan 5y ago
        Yes, but you need to detect the fakes before you can ban them. An adversary could just rotate camera keys before they're even detected.
        sigmoid10 5y ago
        You just need to couple the keys to something tangible. But the truth is that every system can by bypassed if you spend enough effort. The only question is when it becomes enough hassle so that it doesn't disrupt ordinary people's lives. Not even facebook cared about enforcing https until someone made a browser plugin that let everyone steal cookies.
spywaregorilla 5y ago
At best that would allow you to tell if a picture was raw footage. Which is pointless because people edit things all the time.
- telchar 5y ago
  The point is that an image of consequence, if real, would have a signed raw copy available which people could check to verify that there was in fact a real image that showed what a given edited image purported to show. If an image is being circulated but no one as been able to verify that a signed version exists, that image remains untrustworthy. This wouldn't stop people from editing images for aesthetic value.
- roywiggins 5y ago
  I guess there'd have to be certified editors, which would be trusted to allow only "safe" operations on images (crop, white balance, levels, etc) and would sign the resulting jpegs themselves alongside the signature from the camera. It doesn't really seem likely at all to work in practice.
  - spywaregorilla 5y ago
    I want to see an ai that changes picture A to picture B by abusing rounding errors in white balance and levels adjustments.
    - godelski 5y ago
      It isn't exactly what you're looking for, but along the same vein are things called diffusion models.[0] [0] https://lilianweng.github.io/lil-log/2021/07/11/diffusion-models.html https://lilianweng.github.io/lil-log/2021/07/11/diffusion-mo...
    - roywiggins 5y ago
      I think the solution is for the editor to have a hard limit on how many edits it will apply to one image. You'd need to write into the output file how many edits had been performed before signing it and then refuse to process the edited image any further if it's too high. Any scheme like this would be eventually broken somehow, so it's altogether a bad idea imho
arbitrage 5y ago
define 'real camera', please.
- sigmoid10 5y ago
  Nothing stops smartphone or camera manufacturers from adding something like a TPM to the CMOS. In fact, Apple already does sign its hardware components to prevent third party repairs.
smaryjerry 5y ago
Imagine a cryptocurrency like log of data verifying images. Essentially, all cameras will be internet connected with a process to force creation of an NFT at the moment users press the photo button which includes the entire data of the image. Any image without that tech will be considered unreliable. Of course that would have implications that no photo could ever be deleted, I’m guessing this system would be government based and they would be just fine with that.
- rspeele 5y ago
  What can crypto verify here exactly? What stops me from: A. Writing my own software "camera" that does the same gyrations a real camera would do to sign its raw image, but applied to an arbitrary unverified image file on my computer? or B. Printing out an unverified picture and taking a picture of that picture (with good lighting etc so that it is not obvious), with a camera that makes it verified "real"?
  - recursive 5y ago
    A. wouldn't work because there would presumably be a private key or something. B. seems like it would work.
- kkielhofner 5y ago
  This is exactly what my new startup Tovera[0] does. Process is roughly: 1) Upload image with web-based user dashboard. 2) We strip existing metadata (for privacy and all of the reasons everyone else does). 3) Our API generates a unique identifier for the asset (UUIDv4 in hex). 4) The unique identifier is embedded in the image with XMP. 5) A SHA256 checksum of the entire file is generated. 6) Via the API the checksum is associated with the unique identifier (along with some other stuff). 7) (Optionally) the unique identifier and checksum are added as JSON to IPFS via a pinning service and (essentially) an NFT is minted for the JSON verification data on the Polygon blockchain. 8) The user gets sharable[1] and iframe embeddable[2] links and the ability to post directly to various social media networks, etc. When the links are viewed our Javascript reads the unique id, fetches the stored checksum from the API, and generates a new checksum of the image in the browser. If the checksums match a clickable icon appears in the top right of the image with additional information about the image, links to the IPFS and blockchain links, etc. Users can change the additional metadata at anytime and it updates instantly. Long term goals are hosted javascript verification library, browser extensions, mobile SDKs, potential browser/OS integrations, native plugins for popular authoring/editing applications, and so on. Happy to answer any questions! [0] https://tovera.com https://tovera.com [1] https://share.tovera.com/preview/c65b0658ab6e4d89963b1e0a319a1c0a https://share.tovera.com/preview/c65b0658ab6e4d89963b1e0a319... [2] https://share.tovera.com/embed/c65b0658ab6e4d89963b1e0a319a1c0a https://share.tovera.com/embed/c65b0658ab6e4d89963b1e0a319a1...
kkielhofner 5y ago
My new startup Tovera[0] is working on a verification approach that (with some tweaking and user feedback) should be usable by anyone without some of the pesky and potentially privacy concerning aspects of a key/cert based approach. More in my comment here: https://news.ycombinator.com/item?id=28401256#28407184 https://news.ycombinator.com/item?id=28401256#28407184 [0] https://tovera.com https://tovera.com

sideshowb 5y ago

I'm not hopeful that the latter will do much. Just do your deepfake, point a camera at your screen and press the shutter button to get it cryptographically signed..? Ok, so that's tongue in cheek and we'd see some artifacts there but the general principle works. You could intercept the signal from the CCD, or just extract the signing key from the camera's ROM, etc etc.

sigmoid10 5y ago
You can already do that today to fool basic image forensics. Security is never absolute, but if you make the hassle big enough it might suffice to protect the general population. Just like almost all smartphones don't let you mess with the wifi card or spoof your mac address unless you root them. If that wasn't the case, you'd see way more hostapd-wpe attacks these days, since every kid could do it.
- rspeele 5y ago
  In the case of image trustworthiness, it might actually be better for it to be accessible to the general population, if it's doable at all. Then the public knows that images are unreliable evidence.

foobar33333 5y ago

Cryptographic keys embedded in hardware worked really well for the DRM industry. Eventually someone will work out how to extract the key, then create malware to spread around android devices to send back millions of valid keys. What do you do then? Blacklist millions of real peoples cameras and prevent them from using the internet?

sigmoid10 5y ago
You only need a method of key invalidation and renewal, that's all. Ssl certificates have been facing this problem for years. The threat model is equal to someone infecting millions of devices and then sending back banking data, so it's not like people aren't working on mitigating that stuff.
- sebzim4500 5y ago
  >Ssl certificates have been facing this problem for years. That's an entirely different, and much easier problem. In the case of SSL you are not worried about an attacker obtaining a certificate for any host, just the ones you care about. E.g. an attacker getting a certificate for facebook.com would be catastrophic, but an attacker getting a certificate for a website that no one uses would be a non-issue. For the case of avoiding deepfakes, you need to avoid the attacker extracting a key from any of the millions of cameras that are sold every year.
  - uuidgen 5y ago
    Every camera will have a unique key so a single key leaked is still no issue. It's easy to maintain a blacklist - similarly how CRLs work.
    - xrendan 5y ago
      Yes, but you need to detect the fakes before you can ban them. An adversary could just rotate camera keys before they're even detected.
      - sigmoid10 5y ago
        You just need to couple the keys to something tangible. But the truth is that every system can by bypassed if you spend enough effort. The only question is when it becomes enough hassle so that it doesn't disrupt ordinary people's lives. Not even facebook cared about enforcing https until someone made a browser plugin that let everyone steal cookies.

spywaregorilla 5y ago

At best that would allow you to tell if a picture was raw footage. Which is pointless because people edit things all the time.

telchar 5y ago
The point is that an image of consequence, if real, would have a signed raw copy available which people could check to verify that there was in fact a real image that showed what a given edited image purported to show. If an image is being circulated but no one as been able to verify that a signed version exists, that image remains untrustworthy. This wouldn't stop people from editing images for aesthetic value.
roywiggins 5y ago
I guess there'd have to be certified editors, which would be trusted to allow only "safe" operations on images (crop, white balance, levels, etc) and would sign the resulting jpegs themselves alongside the signature from the camera. It doesn't really seem likely at all to work in practice.
- spywaregorilla 5y ago
  I want to see an ai that changes picture A to picture B by abusing rounding errors in white balance and levels adjustments.
  - godelski 5y ago
    It isn't exactly what you're looking for, but along the same vein are things called diffusion models.[0] [0] https://lilianweng.github.io/lil-log/2021/07/11/diffusion-models.html https://lilianweng.github.io/lil-log/2021/07/11/diffusion-mo...
  - roywiggins 5y ago
    I think the solution is for the editor to have a hard limit on how many edits it will apply to one image. You'd need to write into the output file how many edits had been performed before signing it and then refuse to process the edited image any further if it's too high. Any scheme like this would be eventually broken somehow, so it's altogether a bad idea imho

arbitrage 5y ago

define 'real camera', please.

sigmoid10 5y ago
Nothing stops smartphone or camera manufacturers from adding something like a TPM to the CMOS. In fact, Apple already does sign its hardware components to prevent third party repairs.

smaryjerry 5y ago

Imagine a cryptocurrency like log of data verifying images. Essentially, all cameras will be internet connected with a process to force creation of an NFT at the moment users press the photo button which includes the entire data of the image. Any image without that tech will be considered unreliable. Of course that would have implications that no photo could ever be deleted, I’m guessing this system would be government based and they would be just fine with that.

rspeele 5y ago
What can crypto verify here exactly? What stops me from: A. Writing my own software "camera" that does the same gyrations a real camera would do to sign its raw image, but applied to an arbitrary unverified image file on my computer? or B. Printing out an unverified picture and taking a picture of that picture (with good lighting etc so that it is not obvious), with a camera that makes it verified "real"?
- recursive 5y ago
  A. wouldn't work because there would presumably be a private key or something. B. seems like it would work.
kkielhofner 5y ago
This is exactly what my new startup Tovera[0] does. Process is roughly: 1) Upload image with web-based user dashboard. 2) We strip existing metadata (for privacy and all of the reasons everyone else does). 3) Our API generates a unique identifier for the asset (UUIDv4 in hex). 4) The unique identifier is embedded in the image with XMP. 5) A SHA256 checksum of the entire file is generated. 6) Via the API the checksum is associated with the unique identifier (along with some other stuff). 7) (Optionally) the unique identifier and checksum are added as JSON to IPFS via a pinning service and (essentially) an NFT is minted for the JSON verification data on the Polygon blockchain. 8) The user gets sharable[1] and iframe embeddable[2] links and the ability to post directly to various social media networks, etc. When the links are viewed our Javascript reads the unique id, fetches the stored checksum from the API, and generates a new checksum of the image in the browser. If the checksums match a clickable icon appears in the top right of the image with additional information about the image, links to the IPFS and blockchain links, etc. Users can change the additional metadata at anytime and it updates instantly. Long term goals are hosted javascript verification library, browser extensions, mobile SDKs, potential browser/OS integrations, native plugins for popular authoring/editing applications, and so on. Happy to answer any questions! [0] https://tovera.com https://tovera.com [1] https://share.tovera.com/preview/c65b0658ab6e4d89963b1e0a319a1c0a https://share.tovera.com/preview/c65b0658ab6e4d89963b1e0a319... [2] https://share.tovera.com/embed/c65b0658ab6e4d89963b1e0a319a1c0a https://share.tovera.com/embed/c65b0658ab6e4d89963b1e0a319a1...

kkielhofner 5y ago

My new startup Tovera[0] is working on a verification approach that (with some tweaking and user feedback) should be usable by anyone without some of the pesky and potentially privacy concerning aspects of a key/cert based approach. More in my comment here: https://news.ycombinator.com/item?id=28401256#28407184 https://news.ycombinator.com/item?id=28401256#28407184 [0] https://tovera.com https://tovera.com