6 ms·
We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily
by jacobr1 3y ago
We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that can be used by downstream systems to _better_ detect fraud, though not eliminate it.
- sacrosancty 3y ago[dead]
- tudorw 3y ago"Magnetic anomalies are generally a small fraction of the magnetic field. The total field ranges from 25,000 to 65,000 nanoteslas (nT). To measure anomalies, magnetometers need a sensitivity of 10 nT or less." Would signing content with a cryptographically consistent encoding of this field be workable?
- floren 3y agoI was thinking the other day about embedding keys in cameras, etc. but came up with the problem that you could just wire up a computer that BEHAVES like a CCD sensor and send whatever the hell you feel like in to the signing hardware, so you feed in your fake image and it gets signed by the camera as though it were real. I assume smarter people than me have put much more time into the problem, so I'd be interested to see any good resources on the subject.
- jacobr1 3y agoI think you'd need device levels keys. You couldn't trust any particular image ... but you could perhaps know where it came from, which you gives you a better substrate upon which to infer trust.
- tshaddox 3y agoI think you're essentially describing the hardware DRM supply chain. For example, HDCP is a DRM scheme where Intel convinces (or legally requires) every manufacturer of HDMI output devices (e.g. set-top boxes, Blu-ray players) in the world to encrypt certain video streams. Then, Intel requires manufacturers of HDMI input devices (e.g. TVs) to purchase a license key that can decrypt those video streams. This license agreement also requires the manufacturer to design their device such that the device key cannot be easily discovered and the video content cannot be easily copied. Then, Intel gets media companies to include some extra metadata in video media like Blu-ray discs. This metadata can contain revoked device keys, so that if a TV manufacturer violates the terms of the license agreement (e.g. leaks their key, or sells a device that makes copies of video content), that manufacturer's TVs won't be able to play new content that starts including their key in the revocation list. Of course, Intel's HDCP master key was either leaked or reverse-engineered, so anyone can generate their own valid device keys. Intel will probably sue you if you do this, I guess.
- charcircuit 3y ago>Of course, Intel's HDCP master key was either leaked or reverse-engineered, so anyone can generate their own valid device keys Of an older version of HDCP. New media can require a higher HDCP version where that bypass isn't possible.
- tshaddox 3y agoThat's certainly possible, but did that actually happen in practice? The downside is obviously that you essentially start selling new Blu-ray discs that don't work on any old Blu-ray players. I feel like I would have heard if that happened. Unless maybe the old players could issue firmware updates?
- charcircuit 3y ago>That's certainly possible, but did that actually happen in practice? Yes, see 4k blurays for requiring HDCP 2.2 requiring people to get a new bluray player.
- feralderyl 3y agoEvery time I hear about these watermarks that's what I think. There is nothing stopping someone from playing any video on a projector and simply recording the projected video on any device they want.
- mistrial9 3y agoa very large number of consumer and high-end camera equipment does have unique ID of the device. Some metadata stores those device IDs along with other values like mfg or light settings. Its not signed into a binding chain, only marked at image creation time. The vast majority of people I expect would just copy the data files blindly.
- blibble 3y agoyou'd have to do the signing inside the ccd silicon (though if you have lots of time/effort/money you could still extract the key)
- AnthonyMouse 3y agoOr attach the inputs on the CCD silicon to something other than the CCD array.
- atestu 3y agoWhat about adding other sensors like LIDAR to make it harder to fake what you’re filming? I think about it often too…
- gorlilla 3y agoHarder only lasts temporarily until 'simple' can be commoditized. The fact we're discussing this implies there is a market for it since there is a need for any defense.
- yetanotherloss 3y agoThe cryptography to support this has been around for ever and it's been next to impossible to make the decision makers at companies and large organizations care, much less end users. Small time players like GE routinely fail to correctly sign industrial control software, the odds of people recording video paying enough attention to get it right and the meme crowd bothering to check even if they did seems vanishingly small without a lot of educational effort.
- jacobr1 3y agoYeah, you need adoption for it to work, and that in tern means there needs to be some kind of financial or regulatory incentive. But it does seem to me to be more technically feasible. Fingerprinting AI seems ... just not workable at this point. We are starting to see adoption of software supply-chains with SBOMS, albeit imperfectly. We are starting to see increased adoption of things like DMARC in the email space to better authentic the originator of an email. Both are highly imperfect systems ... but you can start kludging something together ... and if the incentives are there I think you can build out more of a workable system.
- tomrod 3y ago> Small time players like GE routinely fail to correctly sign industrial control software, the odds of people recording video paying enough attention to get it right and the meme crowd bothering to check even if they did seems vanishingly small without a lot of educational effort. I've wanted to build a product in this space ever since I heard about deepfakes. Mix of keybase and appropriate file hash, and hash gen for subsets of sections of video. Maybe it needs to be a protocol, maybe a product, not sure, but the need seems apparent to me.
- tornato7 3y agoFor practical uses the cryptography is quite new. Essentially if you cryptographically sign an image coming out of a sensor then you’re only getting whatever original version of it is and if you modify it even just to resize it for web for example, then it’s going to break the cryptographic chain of trust. You need to use zero knowledge proofs to allow for necessary image manipulations while keeping a signature.
- tempusalaria 3y agoYou can see here: GitHub.com/HNx1/IdentityLM It’s a direct (and open source) implementation of public key cryptography into the LLM logit distribution. The paraphrasing model/beam search needs work - feel free to pitch in :)
- nofunphil 3y agoAgreed. At the risk of a shitstorm of downvotes, tokenized media could be part of a solution, especially at the consumer level. Authenticate real videos via a mint button/QR that airdrops you a token from creator. May require platforms to opt-in tho. Basically trust nothing unless you can authenticate source onchain. Not great fo sho, but prob necessary soon
- tornato7 3y agoThis is the best solution IMO - Blockchain allows others to independently verify the time that media was created without needing to trust some random company's database / API in perpetuity.
- Alligaturtle 3y agoI agree with this sentiment. Years ago, I asked around at one of the smartphone companies whether it would be possible to certify to an end user that a photo is either: 1) Authentic and only lightly edited with image manipulation software (e.g., cropped, color balanced, or text placed over top of the image) 2) Produced on a phone that has had to go through hardware hacks Note that the guarantee in (1) wouldn't prevent someone from taking a photo of a TV screen. When I asked that original question, I had quite a few more details about how the certification might be done, how the credentials would be hosted, and how the results would be shown on a website. Anyway, just asking this question was met with a storm of negative responses. I counted two dozen messages that were either neutral (asking for clarification) or else outright hostile before the first hesitantly positive message. My favorite hostile response was that allowing people to certify images as real would steal peoples' rights. I didn't follow the logic, but the guy who made the argument was really into it. There were lots of comments about how using AI would be a better solution, some commenting on how Cannon already did it (and messed up gloriously), others stating they didn't have faith in hardware... it makes a fella never want to ask a question again. In the end, I got an expert to speculate that the technology currently exists, and has existed for 5-10 years, to do this with a modern smartphone. However, unless a high-level engineer or executive argues that providing this feature will somehow be a competitive advantage, there is no appetite to provide this kind of feature.
- not2b 3y agoI could take a photo of someone else's photo with a camera that cryptographically signs the image. Then I suppose I could claim that my photo is the original (see? it is signed, with a camera that maintains a chain of trust) and the original photo is now the stolen one. To pull this off it would have to be a really high quality camera that would make an accurate copy. Perhaps something like this is what your hostile responder was thinking of.
- tornato7 3y agoBut if both images had a timestamp in the signature, wouldn’t you be able to prove that the original was taken first?
- ryukoposting 3y agoI was thinking the same thing- we've had digital signing algorithms for decades, and those seem to work fine enough. There's a healthy distrust of cert authorities, sure, but it still works.
- p1necone 3y agoThe signature only proves that the website/Bill the photographer was involved in the chain sending the website/photograph to you, not anything about the content itself. Unless you have reason to trust Bill himself you can't trust that he actually took the photo, or that it isn't ai generated. Although knowing that Bill isn't tech savvy enough to do those things might be enough.
- jacobr1 3y agoThat ... might be all we get though. If anybody can produce artificial, but realistic content that is indistinguishable from something the real thing, then all we might have is our willingness to trust any given originator/distributor of the content. Narrowing down the problem to "Do we trust Bill" is at least something we can attempt to address. This also eliminates some sources where we can more easily discount things. If the source is 4chan, maybe I don't need much more information to make my repetitional assessment.
- nonameiguess 3y agoWhat's the threat vector you're trying to mitigate here? If you're wondering whether a movie that claims to be produced by Disney really was, if it's in theaters or on Disney+, then you can trust it was actually made by Disney or at least licensed to them. As long as the Washington Post still employs its own photographers and doesn't accept imagery submission from the general public, you should be able to trust a photo published in the Washington Post is at least of something real, unless you just don't trust the Post itself. If you're thinking a YouTube channel or something, unless the channel got hacked, seemingly anything published there was really published by the channel owner. Maybe they're showing you something made by AI that isn't real, but as the owner of their own signing key, nothing would prevent them from signing an AI-generated image. If you're talking someone on Twitter or Facebook is putting a photo in your feed claiming a human photographed BLM throwing bricks through a window, don't trust shit being posted on Facebook or Twitter no matter what, probably, but even there, unless the profile was hacked, you either trust the person who owns it or you don't. Nothing would prevent them from signing a forgery of reality that they legitimately forged themselves. Even with device-level keys, what are you trying to prove? You can pay actors to throw bricks through windows. I guess the concern is this doesn't scale as well as asking Midjourney to do it, but I wonder to what extent that is even true. With 8 billion people on the planet and counting and a whole lot of them doing this shit, given the limited input bandwidth of human sense organs, there is seemingly some maximum saturation of bullshit a person can be exposed to that a lot of people have already hit, and having the Internet host even more of it doesn't mean they'll grow bigger eyes and a faster brain that can actually ingest more bullshit than it already does.
- atestu 3y agoWell, a lot of people don't trust the Washington Post and might want proof that the photos they're posting are of real events. The Washington Post might not trust its photographers completely either (journalists making stuff up happens[0]), so they too might want proof the photos they're getting are real. [0]: https://www.nytimes.com/2003/05/11/us/correcting-the-record-times-reporter-who-resigned-leaves-long-trail-of-deception.html https://www.nytimes.com/2003/05/11/us/correcting-the-record-...
- jallmann 3y agoAgreed that cryptographic provenance seems like the best way forward, however the goal is more about proving authenticity, less about detecting fraud. Small but important distinction. There are industry initiatives around this already such as CAI https://en.m.wikipedia.org/wiki/Content_Authenticity_Initiative https://en.m.wikipedia.org/wiki/Content_Authenticity_Initiat...
- jacobr1 3y agoCAI is roughly the kind of scheme I was suggesting. Thanks for the reference. My take is that proving authenticity might not be something we can do with any degree of accuracy in a general sense. So if that is infeasible, then we need _some_ kind of mitigation. Something like CAI allows us to make the an assessment about the how much trust to give an informational source, probably taking into account multiple factors (known exploits in the source device, reputation of originator and what claims are be attached in the metadata). This might allow me to accept that a given video originated from a local tv station, rather than tiktoker edit, but I still need to asses if that station used genAI or has been compromised or whatever else. But that seems a much narrower reputational problem, that also will be contextual.
- jallmann 3y agoExactly - content authenticity is a narrower and more tractable problem. And I suspect in most cases that is enough - the world runs on trust, so if a reliable source (eg, the NY Times) attests that a photograph is authentic, then we can reasonably trust that. And as you noted, this chain of provenance can go all the way back to the device itself. Fraud, on the other hand, is much harder to prove without a doubt. It is still a problem, but probably less so in the general case [1]. The concerning thing seems to be extremely targeted attacks, eg, hacking a CCTV to implicate someone in a crime. [1] Notwithstanding folks who slurp up unfiltered content on, eg, TikTok, but the most outrageous stuff usually doesn't "make the news" on its own without more vetting. If anything, this will reinforce the importance of actual journalism, fact-checking and corroborating evidence.
- j0hnyl 3y agoThe problem is that this requires a federated digital identity provider which is kind of the antithesis of a utopian internet.
- flextheruler 3y agoWould it really be that different than https? I would assume it would mainly be large entities like governments or corporations utilizing these certificate chains
- AnthonyMouse 3y agoThe certificates for HTTPS certify that you're communicating with someone the CA verified has control over some domain name. They don't require the CA to exercise any control over the endpoint device or for the user to cede any. If you want the camera sensor in your phone to certify that the user hasn't altered the image, the device can't be in the control of the user.