4 ms·

I think we're never going to be able to have robust ai detection, and current models are as bad as they'll ever be. Instead we really need to have the ability t

I think we're never going to be able to have robust ai detection, and current models are as bad as they'll ever be. Instead we really need to have the ability to sign images on cameras that show these are the bits that came off this hardware unedited, that professional news outlets can verify. But that's going to cost money to make and market all these new cameras and I just don't know how we incentivize or pay for this, so we're left unable to trust any images and video in the near future. I can only think of technical solutions and not the social changes that need to happen before the tech is wanted and adopted.

breve 9mo ago
Sony cameras can sign still images and videos to vouch that they are not AI generated: https://authenticity.sony.net/camera/en-us/index.html https://authenticity.sony.net/camera/en-us/index.html https://www.sony.eu/presscentre/sony-launches-camera-verify-feature-for-its-camera-authenticity-solution-for-news-organisations https://www.sony.eu/presscentre/sony-launches-camera-verify-... Ideally it'd become an open standard supported by all manufacturers. Which is what they're trying to do: https://c2pa.org/ https://c2pa.org/
- mattnewton 9mo ago
  Thank you, this is fantastic to know! I think we have to normalize requiring this or similar standards for news, it will go a long way. Ideally we would have a similar attestation from most people's cameras (on their smartphones) but that's a much harder problem to also support with 3p camera apps.
  - 2OEH8eoCRo0 9mo ago
    More like I won't trust anything that doesn't come from a press photographer.
    - cmxch 9mo ago
      And what will make them more trustworthy?
      - ndsipa_pomu 9mo ago
        Their career prospects would vanish if caught doctoring images with AI. Unfortunately, the same can't be said of governmental employees.
  - 93po 9mo ago
    it doesnt really matter if you can just take a photo of an AI image that's been printed out
    - mattnewton 9mo ago
      That will look like a photo of a printout though. Seems easier to just hack the hardware to get it to sign arbitrary images instead.
      - 93po 9mo ago
        ok but then the conversation switches from "was this actually taken from a camera" to "is this a photo of a printout" and we're not really any further along in being able to establish trust in what we're seeing, my point is the goal posts will always get moved because unless we see literally anything in person these days, we can't really trust in it
direwolf20 9mo ago
Then you can have a signed picture of a screen showing an AI image. And the government will have a secret version of OpenAI that has a camera signature.
throwaway89201 9mo ago
This sounds like a good idea on its face, but it will have the effect of both legitimizing altered photos and delegitimizing photos of actual events. You will need camera DRM with a hardware security module down all the way to the image sensor, where the hardware is in the hands of the attacker. Even when that chain is unbroken, you'll need to detect all kinds of tricks where the incoming photons themselves are altered. In the simplest case: a photo of a photo. If HDCP has taught anything, it's that vendors of consumer products cannot implement such a secure chain at all, with ridiculous security vulnerabilities for years. HDCP has been given up and has become mostly irrelevant, perhaps except for the criminal liability it places on 'breaking' it. Vendors are also pushed to rely on security by obscurity, which will make such vulnerabilities harder to find for researchers than for attackers. If you have half of such a 'signed photos' system in place, it will become easier to dismiss photos of actual events on the basis that they're unsigned. If a camera model or security chip shared by many models turns out to be broken, or a new photo-of-a-photo trick becomes known, a huge amount of photos produced before that, become immediately suspect. If you gatekeep (the proper implementations of) these features only to professional or expensive models, citizen journalism will be disincentivized. But even more importantly: if you choose to rely on technical measures that are poorly understood by the general public (and that are likely to blow up in your face), you erode a social system of trust that already is in place, which is journalism. Although the rise of social media, illiteracy and fascism tends to suggest otherwise, journalistic chain of custody of photographic records mainly works fine. But only if we keep maintaining and teaching that system.
deleted 9mo ago
[deleted]
datsci_est_2015 9mo ago
But especially when a party has been shown to alter photos with evidence even for “memetic” reasons, they’ve poisoned their own reliability. As far as I’m concerned the DOJ us no longer a reliable source of evidence until a serious purge of leadership due to their intimate connection with the parties who posted this edited photo.

breve 9mo ago

Sony cameras can sign still images and videos to vouch that they are not AI generated: https://authenticity.sony.net/camera/en-us/index.html https://authenticity.sony.net/camera/en-us/index.html https://www.sony.eu/presscentre/sony-launches-camera-verify-feature-for-its-camera-authenticity-solution-for-news-organisations https://www.sony.eu/presscentre/sony-launches-camera-verify-... Ideally it'd become an open standard supported by all manufacturers. Which is what they're trying to do: https://c2pa.org/ https://c2pa.org/

mattnewton 9mo ago
Thank you, this is fantastic to know! I think we have to normalize requiring this or similar standards for news, it will go a long way. Ideally we would have a similar attestation from most people's cameras (on their smartphones) but that's a much harder problem to also support with 3p camera apps.
- 2OEH8eoCRo0 9mo ago
  More like I won't trust anything that doesn't come from a press photographer.
  - cmxch 9mo ago
    And what will make them more trustworthy?
    - ndsipa_pomu 9mo ago
      Their career prospects would vanish if caught doctoring images with AI. Unfortunately, the same can't be said of governmental employees.
- 93po 9mo ago
  it doesnt really matter if you can just take a photo of an AI image that's been printed out
  - mattnewton 9mo ago
    That will look like a photo of a printout though. Seems easier to just hack the hardware to get it to sign arbitrary images instead.
    - 93po 9mo ago
      ok but then the conversation switches from "was this actually taken from a camera" to "is this a photo of a printout" and we're not really any further along in being able to establish trust in what we're seeing, my point is the goal posts will always get moved because unless we see literally anything in person these days, we can't really trust in it

direwolf20 9mo ago

Then you can have a signed picture of a screen showing an AI image. And the government will have a secret version of OpenAI that has a camera signature.

throwaway89201 9mo ago

This sounds like a good idea on its face, but it will have the effect of both legitimizing altered photos and delegitimizing photos of actual events. You will need camera DRM with a hardware security module down all the way to the image sensor, where the hardware is in the hands of the attacker. Even when that chain is unbroken, you'll need to detect all kinds of tricks where the incoming photons themselves are altered. In the simplest case: a photo of a photo. If HDCP has taught anything, it's that vendors of consumer products cannot implement such a secure chain at all, with ridiculous security vulnerabilities for years. HDCP has been given up and has become mostly irrelevant, perhaps except for the criminal liability it places on 'breaking' it. Vendors are also pushed to rely on security by obscurity, which will make such vulnerabilities harder to find for researchers than for attackers. If you have half of such a 'signed photos' system in place, it will become easier to dismiss photos of actual events on the basis that they're unsigned. If a camera model or security chip shared by many models turns out to be broken, or a new photo-of-a-photo trick becomes known, a huge amount of photos produced before that, become immediately suspect. If you gatekeep (the proper implementations of) these features only to professional or expensive models, citizen journalism will be disincentivized. But even more importantly: if you choose to rely on technical measures that are poorly understood by the general public (and that are likely to blow up in your face), you erode a social system of trust that already is in place, which is journalism. Although the rise of social media, illiteracy and fascism tends to suggest otherwise, journalistic chain of custody of photographic records mainly works fine. But only if we keep maintaining and teaching that system.

deleted 9mo ago

[deleted]

datsci_est_2015 9mo ago

But especially when a party has been shown to alter photos with evidence even for “memetic” reasons, they’ve poisoned their own reliability. As far as I’m concerned the DOJ us no longer a reliable source of evidence until a serious purge of leadership due to their intimate connection with the parties who posted this edited photo.