5 ms·

Leica camera has built-in defense against misleading AI, costs $9,125

41 points by nathandaly 3y ago

bberenberg 3y ago
Not sure how this kind of thing works given that metadata can be stripped. Would love if the article further explained that side of things.
- firedaemon 3y ago
  I think the idea is simply that you can prove authenticity if you have the metadata. You can’t prove authenticity without metadata, nor can you prove “inauthenticity” without metadata.
  - deleted 3y ago
    [deleted]
- hooverd 3y ago
  It's like a signed message. The image is the clear text. The point is verifying authenticity.
traverseda 3y ago
You can probably build a rig to feed false data to the sensor, or take a photo of a sufficiently high-resolution display, or use side-channel power analysis to extract the cryptographic keys. I have serious doubts this kind of provenance metadata will actually work against a sophisticated attacker (someone willing to spend more than 100k on making fake images), but I'm sure it will be used as an excuse to further lock down computing platforms.
- roomey 3y ago
  Raising the bar of fakes is probably the goal with this
- generj 3y ago
  There are probably many possible methods to detect a picture being of a display rather than the real world. The easiest clues are depth of field and focal length metadata mismatching (presumably a macro lens or similar would be needed, and also presuming lens metadata is not spoofable and included in the signed metadata). Color accuracy will be subtly off. Cameras can detect more dynamic range depth than most (any) screens can create particularly for outdoor scenes. That’s just what I can come up with right now - scientists will have a field day publishing techniques to detect pictures of screens. Inserting fake sensor readings is plausible but complicated enough I doubt any but state actors would or could bother. Even this standard only raises the cost to create fake images to 100K would be fantastic for journalism and democracy. Much better than the cost being $0.
  - jychang 3y ago
    Typical tech response that ignores the realityy of users: Most leica lenses predate electronics. Lol. What DoF/focal length metadata?
- illwrks 3y ago
  This is probably useful in a legal context more than anything else. Imagine photos presented as evidence in court... photos taken with a known 'approved' device, with trusted credentials and a way to prove the photos came from said device. I'm sure there are large government contracts for such a device.
- great_psy 3y ago
  I think each camera has its own key. So to do any side channel attack you would need to steal the camera, in which case you can just take a picture of a printout in a lab and then it’s all signed by the camera. The signature just says who took the picture, not that the image is not ai generated.
- M95D 3y ago
  > feed false data to the sensor Then why not replace the sensor entirely and send fake sensor data? It would be difficult to fake depth of field changes in response to camera's focus motor moving. If I were a security researcher, I would really try to see if the camera is smart enough to tell if the sensor data is fake. > take a photo of a sufficiently high-resolution display A 60 MP display? (10000x6000 resolution? I want one!!!) That would still make pixels visible. I suspect a display at least 4x larger would be needed. And it needs to be curved, or the out-of-focus corners will ruin the illusion.
  - M95D 3y ago
    I was wrong about the curved display and focus. Sorry.
chrisa 3y ago
Worth noting that the expensive price isn't because of the metadata feature - it's a new version of the Leica M11 which is $8995 itself (Leicas are just crazy expensive)
- __loam 3y ago
  Yeah this isn't something to get outraged over. The arguably "entry level" leica is over $6k. They're expensive but well made cameras that are hand built in Germany. If you're buying one you probably know what you're getting or you have more money than you know what to do with. If you think they're expensive, there's awesome options from fujifilm, Canon, nikon, and Sony that cost less.
- vr46 3y ago
  The non-Ms are beautifully made and worth every penny (eg Q2, SL) but the Ms are beautifully made terrible technology - specifically the rangefinder: they’re delicate, need frequent servicing and can only focus down to 0.7m, so the advent of live view finally allows lenses that go closer than 0.7m to be focused. That said, I just bought a Canon 7D from 2009 and it too is beautifully made. However, I think the digital signing is a good way to provide the photographer with proof that the picture in the paper/competition is the picture that they made with the camera and not their pal Midjourney. Why they put it in such a sluggish contraption, I don’t know, but probably a good place to start compared to a Z9 and trying to tag pictures at 500 frames a second or whatever.
roomey 3y ago
I can imagine this becoming pretty standard in the near term, particularly in new smart phones, and also done for video too. As ever, yes it can be faked (by taking A picture of a screen for example) but it ups the cost of faking a massive amount from where it is currently
deleted 3y ago
[deleted]
MarkusWandel 3y ago
"Whenever someone subsequently edits that photo, the changes are recorded to an updated manifest, rebundled with the image, and updated in the Content Credentials database whenever it is reshared on social media. Users who find these images online can click on the CR icon in the [pictures'] corner to pull up all of this historical manifest information as well, providing a clear chain of providence, presumably, all the way back to the original photographer." Fat chance. I can think of exactly zero examples where a photo shared on social media, or even on Whatsapp, has its metadata intact. This is frustrating to me because often it's the only way to get a photo from computer illiterates, and I like my EXIF data, specifically, the exact time/date the picture was taken.
- MarkusWandel 3y ago
  Of course metadata stripping protects people's privacy, since so many photos are geotagged these days. But still frustrating.
- deleted 3y ago
  [deleted]
- ncr100 3y ago
  Shall we begin to expect provenance, then? Like, for our own factual informational health?
hooverd 3y ago
Huh, there has to be a better way to do this than with a proprietary SaaS service.
- mhitza 3y ago
  Of course, allowing consumers to push their custom certs to the chip, without "rooting", instead of having central control.
  - lxgr 3y ago
    The point here is that the photographer is not necessarily trusted. You can already sign a JPEG with your GPG key; that won’t convince anyone that it wasn’t actually photoshopped or outright generated by AI. The point here is to have a more trusted hardware vendor vouch for their camera not being easy to trick into signing arbitrary data, but only actual images it took itself. Of course that also puts a lot of pressure on the key generation, storage and processing mechanism of that vendor; trusted computing in this scenario (i.e. the adversary has unrestricted and persistent access to the system) isn’t easy to get right.
wenc 3y ago
One issue is because there are so few CR cameras out there, most people would sign their photos in Lightroom. So the provenance starts from software rather than capture device. So you can sign a fake photo that you modified and its provenance from that point on is traceable. But you still wouldn’t know if the photo was captured authentically. I guess it doesn’t matter — this is about traceability rather than authenticity.
- lxgr 3y ago
  In this use case, it’s arguably the other way around: The point is to prove that the image was taken in the real world, to stakeholders that might not necessarily take your word for it otherwise.
FireBeyond 3y ago
Content Credentials is just one/the latest standard. Canon (maybe others) offered a similar feature from the very early days of digital, with a module for the 1Dx at least that would cryptographically sign files the camera generated to say "this is what the sensor saw". Typically it was marketed to law enforcement, because there was a wariness around digital photography as evidence even in the early days.
onetimeuse92304 3y ago
There is one hitch. What if AI-generated content is projected through camera lens? That is pretty simple to do.
- lxgr 3y ago
  Without any visible artifacts? I would imagine that to be quite difficult actually!
  - onetimeuse92304 3y ago
    Cameras are used to scan analog film. I have used it in the past, it works perfectly and there is no way for you to tell it was scanned this way. But here is an alternative: You pass an image through a lens, you take photo from the camera. Now, having the input original AI-generated image and photo with the artifacts you calculate what changes your camera introduces. You reverse it to know what needs to be modified in original image to get photo exactly as you want. If one pass does not suffice, you do multiple passes. Yes, it is easy, as in a CS or math student can do it.
    - M95D 3y ago
      How would you display the fake image in front of the camera? That camera has a 60 MP sensor. That about 10'000 x 6'000 pixels. Eight times higher than a 8K display! Also, you would need a curved spherical display, or the camera won't be able to focus on it's center and corners at the same time. What photographic analog film has at least a 60 MP resolution without any blur or dim colors? How would you transfer a digital fake image on that film? How would you take a picture of that film without losing even more resolution, color, sharpness?
      - onetimeuse92304 3y ago
        First, you mistakenly assume every photo of note is sharp. That's absolutely not true. Second, no you do not need a spherical display. Depth of field IS A PLANE. Also, photographing analog film is done from good enough distance to get rid of any curvature for the practical purposes. Third, photographic analog films can have hundreds of MPs. But it does not matter, the choice of medium you present to the camera is yours and it does not have to be analog film. I am just giving analog film as an example because it is high resolution compared to even best prints or digital displays. Fourth. Before you start saying something is not possible, would you mind to do an even most basic cursory search? Here is an example of people doing this professionally with a DSLR: https://www.youtube.com/watch?v=BVmIhwG0XfU https://www.youtube.com/watch?v=BVmIhwG0XfU
petabytes 3y ago
About a year ago when AI was getting really big, I was really interested in C2PA, since then I've come to the conclusion that actual artists don't care about any of that crap
- nvm0n2 3y ago
  Yeah I've also looked into this, even designed ways to do it with an iPhone (which has a depth camera useful to detect pictures of screens). But despite all the bellyaching over AI and even finding websites that are clearly suffering from fake AI generated profiles, there doesn't seem to be much interest in solutions.
valdiorn 3y ago
Uh, deja vu? Didn't Canon have a system very similar to this, and it famously got cracked over a decade ago? https://it.slashdot.org/story/10/12/03/2133218/canons-image-verification-system-cracked https://it.slashdot.org/story/10/12/03/2133218/canons-image-...
rediguanayum 3y ago
Content credentials (https://contentauthenticity.org/ https://contentauthenticity.org/) appears to use a Certificate Authority (CA) system to authenticate information, and they believe it can help fight misinformation: https://rd.nytimes.com/projects/using-secure-sourcing-to-combat-misinformation https://rd.nytimes.com/projects/using-secure-sourcing-to-com... Anyone here has insight into the PKI behind it?
RecycledEle 3y ago
Digital signing by cameras would be great if only the government and the corporations did not have the private keys.

41 points by nathandaly 3y ago

Not sure how this kind of thing works given that metadata can be stripped. Would love if the article further explained that side of things.

firedaemon 3y ago
I think the idea is simply that you can prove authenticity if you have the metadata. You can’t prove authenticity without metadata, nor can you prove “inauthenticity” without metadata.
- deleted 3y ago
  [deleted]
hooverd 3y ago
It's like a signed message. The image is the clear text. The point is verifying authenticity.

traverseda 3y ago

You can probably build a rig to feed false data to the sensor, or take a photo of a sufficiently high-resolution display, or use side-channel power analysis to extract the cryptographic keys. I have serious doubts this kind of provenance metadata will actually work against a sophisticated attacker (someone willing to spend more than 100k on making fake images), but I'm sure it will be used as an excuse to further lock down computing platforms.

roomey 3y ago
Raising the bar of fakes is probably the goal with this
generj 3y ago
There are probably many possible methods to detect a picture being of a display rather than the real world. The easiest clues are depth of field and focal length metadata mismatching (presumably a macro lens or similar would be needed, and also presuming lens metadata is not spoofable and included in the signed metadata). Color accuracy will be subtly off. Cameras can detect more dynamic range depth than most (any) screens can create particularly for outdoor scenes. That’s just what I can come up with right now - scientists will have a field day publishing techniques to detect pictures of screens. Inserting fake sensor readings is plausible but complicated enough I doubt any but state actors would or could bother. Even this standard only raises the cost to create fake images to 100K would be fantastic for journalism and democracy. Much better than the cost being $0.
- jychang 3y ago
  Typical tech response that ignores the realityy of users: Most leica lenses predate electronics. Lol. What DoF/focal length metadata?
illwrks 3y ago
This is probably useful in a legal context more than anything else. Imagine photos presented as evidence in court... photos taken with a known 'approved' device, with trusted credentials and a way to prove the photos came from said device. I'm sure there are large government contracts for such a device.
great_psy 3y ago
I think each camera has its own key. So to do any side channel attack you would need to steal the camera, in which case you can just take a picture of a printout in a lab and then it’s all signed by the camera. The signature just says who took the picture, not that the image is not ai generated.
M95D 3y ago
> feed false data to the sensor Then why not replace the sensor entirely and send fake sensor data? It would be difficult to fake depth of field changes in response to camera's focus motor moving. If I were a security researcher, I would really try to see if the camera is smart enough to tell if the sensor data is fake. > take a photo of a sufficiently high-resolution display A 60 MP display? (10000x6000 resolution? I want one!!!) That would still make pixels visible. I suspect a display at least 4x larger would be needed. And it needs to be curved, or the out-of-focus corners will ruin the illusion.
- M95D 3y ago
  I was wrong about the curved display and focus. Sorry.

chrisa 3y ago

Worth noting that the expensive price isn't because of the metadata feature - it's a new version of the Leica M11 which is $8995 itself (Leicas are just crazy expensive)

__loam 3y ago
Yeah this isn't something to get outraged over. The arguably "entry level" leica is over $6k. They're expensive but well made cameras that are hand built in Germany. If you're buying one you probably know what you're getting or you have more money than you know what to do with. If you think they're expensive, there's awesome options from fujifilm, Canon, nikon, and Sony that cost less.
vr46 3y ago
The non-Ms are beautifully made and worth every penny (eg Q2, SL) but the Ms are beautifully made terrible technology - specifically the rangefinder: they’re delicate, need frequent servicing and can only focus down to 0.7m, so the advent of live view finally allows lenses that go closer than 0.7m to be focused. That said, I just bought a Canon 7D from 2009 and it too is beautifully made. However, I think the digital signing is a good way to provide the photographer with proof that the picture in the paper/competition is the picture that they made with the camera and not their pal Midjourney. Why they put it in such a sluggish contraption, I don’t know, but probably a good place to start compared to a Z9 and trying to tag pictures at 500 frames a second or whatever.

roomey 3y ago

I can imagine this becoming pretty standard in the near term, particularly in new smart phones, and also done for video too. As ever, yes it can be faked (by taking A picture of a screen for example) but it ups the cost of faking a massive amount from where it is currently

deleted 3y ago

[deleted]

MarkusWandel 3y ago

"Whenever someone subsequently edits that photo, the changes are recorded to an updated manifest, rebundled with the image, and updated in the Content Credentials database whenever it is reshared on social media. Users who find these images online can click on the CR icon in the [pictures'] corner to pull up all of this historical manifest information as well, providing a clear chain of providence, presumably, all the way back to the original photographer." Fat chance. I can think of exactly zero examples where a photo shared on social media, or even on Whatsapp, has its metadata intact. This is frustrating to me because often it's the only way to get a photo from computer illiterates, and I like my EXIF data, specifically, the exact time/date the picture was taken.

MarkusWandel 3y ago
Of course metadata stripping protects people's privacy, since so many photos are geotagged these days. But still frustrating.
deleted 3y ago
[deleted]
ncr100 3y ago
Shall we begin to expect provenance, then? Like, for our own factual informational health?

hooverd 3y ago

Huh, there has to be a better way to do this than with a proprietary SaaS service.

mhitza 3y ago
Of course, allowing consumers to push their custom certs to the chip, without "rooting", instead of having central control.
- lxgr 3y ago
  The point here is that the photographer is not necessarily trusted. You can already sign a JPEG with your GPG key; that won’t convince anyone that it wasn’t actually photoshopped or outright generated by AI. The point here is to have a more trusted hardware vendor vouch for their camera not being easy to trick into signing arbitrary data, but only actual images it took itself. Of course that also puts a lot of pressure on the key generation, storage and processing mechanism of that vendor; trusted computing in this scenario (i.e. the adversary has unrestricted and persistent access to the system) isn’t easy to get right.

wenc 3y ago

One issue is because there are so few CR cameras out there, most people would sign their photos in Lightroom. So the provenance starts from software rather than capture device. So you can sign a fake photo that you modified and its provenance from that point on is traceable. But you still wouldn’t know if the photo was captured authentically. I guess it doesn’t matter — this is about traceability rather than authenticity.

lxgr 3y ago
In this use case, it’s arguably the other way around: The point is to prove that the image was taken in the real world, to stakeholders that might not necessarily take your word for it otherwise.

FireBeyond 3y ago

Content Credentials is just one/the latest standard. Canon (maybe others) offered a similar feature from the very early days of digital, with a module for the 1Dx at least that would cryptographically sign files the camera generated to say "this is what the sensor saw". Typically it was marketed to law enforcement, because there was a wariness around digital photography as evidence even in the early days.

onetimeuse92304 3y ago

There is one hitch. What if AI-generated content is projected through camera lens? That is pretty simple to do.

lxgr 3y ago
Without any visible artifacts? I would imagine that to be quite difficult actually!
- onetimeuse92304 3y ago
  Cameras are used to scan analog film. I have used it in the past, it works perfectly and there is no way for you to tell it was scanned this way. But here is an alternative: You pass an image through a lens, you take photo from the camera. Now, having the input original AI-generated image and photo with the artifacts you calculate what changes your camera introduces. You reverse it to know what needs to be modified in original image to get photo exactly as you want. If one pass does not suffice, you do multiple passes. Yes, it is easy, as in a CS or math student can do it.
  - M95D 3y ago
    How would you display the fake image in front of the camera? That camera has a 60 MP sensor. That about 10'000 x 6'000 pixels. Eight times higher than a 8K display! Also, you would need a curved spherical display, or the camera won't be able to focus on it's center and corners at the same time. What photographic analog film has at least a 60 MP resolution without any blur or dim colors? How would you transfer a digital fake image on that film? How would you take a picture of that film without losing even more resolution, color, sharpness?
    - onetimeuse92304 3y ago
      First, you mistakenly assume every photo of note is sharp. That's absolutely not true. Second, no you do not need a spherical display. Depth of field IS A PLANE. Also, photographing analog film is done from good enough distance to get rid of any curvature for the practical purposes. Third, photographic analog films can have hundreds of MPs. But it does not matter, the choice of medium you present to the camera is yours and it does not have to be analog film. I am just giving analog film as an example because it is high resolution compared to even best prints or digital displays. Fourth. Before you start saying something is not possible, would you mind to do an even most basic cursory search? Here is an example of people doing this professionally with a DSLR: https://www.youtube.com/watch?v=BVmIhwG0XfU https://www.youtube.com/watch?v=BVmIhwG0XfU

petabytes 3y ago

About a year ago when AI was getting really big, I was really interested in C2PA, since then I've come to the conclusion that actual artists don't care about any of that crap

nvm0n2 3y ago
Yeah I've also looked into this, even designed ways to do it with an iPhone (which has a depth camera useful to detect pictures of screens). But despite all the bellyaching over AI and even finding websites that are clearly suffering from fake AI generated profiles, there doesn't seem to be much interest in solutions.

valdiorn 3y ago

Uh, deja vu? Didn't Canon have a system very similar to this, and it famously got cracked over a decade ago? https://it.slashdot.org/story/10/12/03/2133218/canons-image-verification-system-cracked https://it.slashdot.org/story/10/12/03/2133218/canons-image-...

rediguanayum 3y ago

Content credentials (https://contentauthenticity.org/ https://contentauthenticity.org/) appears to use a Certificate Authority (CA) system to authenticate information, and they believe it can help fight misinformation: https://rd.nytimes.com/projects/using-secure-sourcing-to-combat-misinformation https://rd.nytimes.com/projects/using-secure-sourcing-to-com... Anyone here has insight into the PKI behind it?

RecycledEle 3y ago

Digital signing by cameras would be great if only the government and the corporations did not have the private keys.