3 ms·

Maybe it was just the samples that I got, but I found the audio fakes extremely unconvincing - I think they sounded not much different than some comedian trying

by xg15 5y ago

Maybe it was just the samples that I got, but I found the audio fakes extremely unconvincing - I think they sounded not much different than some comedian trying to do an impression of them. I found the silent videos much harder, though more in the direction that I marked a lot of them as fake when they were in fact real. I didn't get the reasoning behind the text-only examples at all. It's impossible to verify a quote just by looking at it. We know this for several centuries - that's why academia makes such a big deal of tracking sources. So what were they trying to do here? In general, I think the performance results here might be a bit too optimistic: It's one thing to spot a fake if you are told to specifically look for them. However, if one of those were just casually embedded in some news report while the attention of the audience is somewhere completely different, I doubt we could spot them as easily. Even moreso if the fake isn't of the two most well-known faces of the western world.

prox 5y ago
I wonder if this is a good (alternatIve) use for a blockchain. If I know address X is from the whitehouse, and they post on that address and it gets sourced from that address for distribution, wouldn’t that be a certification of its origin?
- mehrdada 5y ago
  If you require knowing "an address belongs to whitehouse", why not just serve some content on that "address" (let's say "whitehouse.gov"). Or just sign a message and serve the signed message. Oh no... that nail did not require the blockchain hammer.
  - prox 5y ago
    I was thinking more of seeing a video and the being able to verify it with the original source. Like a hash.
    - mehrdada 5y ago
      Did not mean to be dismissive of your point. It is just that the overhang of blockchain as a solution to everything is so prevalent that sometimes need intervention. It is true that the cryptographic primitives that blockchain is built on are relevant to many other things that we can do and have done. Blockchain in fact utilizes those properties to build a distributed shared state on which the parties have consensus. That, however, is a very special use case of those primitives. If you need message authentication, digital signatures have done that for decades and you don't need an expensive associated database to accomplish that.
    - archi42 5y ago
      It could also be signed using SSL or GPG or really just any other kind of PKI. In the end I need to establish a root of trust, so I either way I need to know that 2c414...2d7 is indeed the hash of the chain_address/ssl_cert/root_ca/gpg_key/... - so in that regard there is no advantage of using a blockchain over a "traditional" PKI. Amend: The real question is how to verify cropped and recompressed snippets. In academia or HN we add a "see foo et al. [1]", so probably that needs to be encoded into the data stream, e.g. "authoritative src is https://media.whitehouse.gov/permanent-url/1234 https://media.whitehouse.gov/permanent-url/1234[...] with hash $hash signed by $cert_hash on $data, portions are $some_encoding_here" and better be signed as well.
- krisoft 5y ago
  I don’t think so. It is trying to solve the wrong problem, and the problem it manages to solve has simpler solution already. You are solving the problem where an entity (the whitehouse) wants you to know that a message came from them. Yes they can use blockchain to sign it, but they can also just put it on their website, invite the press pool for an announcement, and have their PR folks reiterate the message every so often anyone asks. This is not the problem with deepfakes. Imagine that a harmfull footage of the president saying something terrible emerges and someone has to decide if it is real or fake. This someone might be a simple citizen, a journalist, an editor etc. They check the blockchain and see that it is not signed by the whitehouse. Does that mean that it is fake? Maybe, maybe not. Maybe the president just wants to surpress it. Maybe it was an off the cuff remark while they thought nobody is recording them, or maybe they realised that they made a mistake and decided to not sign the footage after all. Which means that the person judging the authenticity of the footage haven’t gained anything from checking the blockchain. Just as an example: during the Trump presidential campaing the now infamous “grab them by the p*sy” footage emerged. Obviously it wasn’t an official endorsed message from his campaign. Would you say it was fake just because it wasn’t signed by them? So how would this blockchain thing help exactly?
  - prox 5y ago
    Good points and you raise some issues I hadn’t considered. I honestly don’t know :) My idea of the blockchain is that it is a chain that could be a chain of trust. Since it seems it is hard to temper with whoever owns an endpoint. But like you say there are other means for that.
phire 5y ago
> I didn't get the reasoning behind the text-only examples at all. So the researchers can quantify how much of people's ablitity to detect deep fakes is coming from the video/audio and not the textual content and knowledge of politician's actual political position.
Etherlord87 5y ago
I also didn't find the videos hard, as lip movement looked quite fake. Maybe for some people they would fit better, but both Trump and Biden are too old to move their lips so vigorously. As for the text: you can easily have more than 50% success rate: - with political knowledge you know some things are unlikely to be said by a given politician - ironically, if the text is grammatically correct, it's unlikely that it was actually spoken by a politician - real transcripts have stuttering. Of course AI could be train to contain stuttering as well, or a bad AI could spew out complete garbage. I think in case of both Biden and Trump that makes it especially hard to differentiate between bad AI and them :D.
- xg15 5y ago
  Even a genuine quote can be presented out of context. For an example that is a bit on the nose, I recently read a quote from Sebastian Haffner about German history, that went like this: > Germany was a much happier place at the turn of the 20th century than it was in the 80s. Which sounds odd, coming from a left-wing, liberal journalist - unless you know from context that he was talking about the 1880s, not the 1980s. He was not wishing the Kaiserreich back, he was just comparing different historical phases of it. Nevertheless, without the context, the quote would look highly strange, even if it is technically not a fake.
dwighttk 5y ago
They were isolating the various aspects one might use to detect fakes. Audio only, video only, political content only.

by xg15 5y ago

prox 5y ago

I wonder if this is a good (alternatIve) use for a blockchain. If I know address X is from the whitehouse, and they post on that address and it gets sourced from that address for distribution, wouldn’t that be a certification of its origin?

mehrdada 5y ago
If you require knowing "an address belongs to whitehouse", why not just serve some content on that "address" (let's say "whitehouse.gov"). Or just sign a message and serve the signed message. Oh no... that nail did not require the blockchain hammer.
- prox 5y ago
  I was thinking more of seeing a video and the being able to verify it with the original source. Like a hash.
  - mehrdada 5y ago
    Did not mean to be dismissive of your point. It is just that the overhang of blockchain as a solution to everything is so prevalent that sometimes need intervention. It is true that the cryptographic primitives that blockchain is built on are relevant to many other things that we can do and have done. Blockchain in fact utilizes those properties to build a distributed shared state on which the parties have consensus. That, however, is a very special use case of those primitives. If you need message authentication, digital signatures have done that for decades and you don't need an expensive associated database to accomplish that.
  - archi42 5y ago
    It could also be signed using SSL or GPG or really just any other kind of PKI. In the end I need to establish a root of trust, so I either way I need to know that 2c414...2d7 is indeed the hash of the chain_address/ssl_cert/root_ca/gpg_key/... - so in that regard there is no advantage of using a blockchain over a "traditional" PKI. Amend: The real question is how to verify cropped and recompressed snippets. In academia or HN we add a "see foo et al. [1]", so probably that needs to be encoded into the data stream, e.g. "authoritative src is https://media.whitehouse.gov/permanent-url/1234 https://media.whitehouse.gov/permanent-url/1234[...] with hash $hash signed by $cert_hash on $data, portions are $some_encoding_here" and better be signed as well.
krisoft 5y ago
I don’t think so. It is trying to solve the wrong problem, and the problem it manages to solve has simpler solution already. You are solving the problem where an entity (the whitehouse) wants you to know that a message came from them. Yes they can use blockchain to sign it, but they can also just put it on their website, invite the press pool for an announcement, and have their PR folks reiterate the message every so often anyone asks. This is not the problem with deepfakes. Imagine that a harmfull footage of the president saying something terrible emerges and someone has to decide if it is real or fake. This someone might be a simple citizen, a journalist, an editor etc. They check the blockchain and see that it is not signed by the whitehouse. Does that mean that it is fake? Maybe, maybe not. Maybe the president just wants to surpress it. Maybe it was an off the cuff remark while they thought nobody is recording them, or maybe they realised that they made a mistake and decided to not sign the footage after all. Which means that the person judging the authenticity of the footage haven’t gained anything from checking the blockchain. Just as an example: during the Trump presidential campaing the now infamous “grab them by the p*sy” footage emerged. Obviously it wasn’t an official endorsed message from his campaign. Would you say it was fake just because it wasn’t signed by them? So how would this blockchain thing help exactly?
- prox 5y ago
  Good points and you raise some issues I hadn’t considered. I honestly don’t know :) My idea of the blockchain is that it is a chain that could be a chain of trust. Since it seems it is hard to temper with whoever owns an endpoint. But like you say there are other means for that.

phire 5y ago

> I didn't get the reasoning behind the text-only examples at all. So the researchers can quantify how much of people's ablitity to detect deep fakes is coming from the video/audio and not the textual content and knowledge of politician's actual political position.

Etherlord87 5y ago

I also didn't find the videos hard, as lip movement looked quite fake. Maybe for some people they would fit better, but both Trump and Biden are too old to move their lips so vigorously. As for the text: you can easily have more than 50% success rate: - with political knowledge you know some things are unlikely to be said by a given politician - ironically, if the text is grammatically correct, it's unlikely that it was actually spoken by a politician - real transcripts have stuttering. Of course AI could be train to contain stuttering as well, or a bad AI could spew out complete garbage. I think in case of both Biden and Trump that makes it especially hard to differentiate between bad AI and them :D.

xg15 5y ago
Even a genuine quote can be presented out of context. For an example that is a bit on the nose, I recently read a quote from Sebastian Haffner about German history, that went like this: > Germany was a much happier place at the turn of the 20th century than it was in the 80s. Which sounds odd, coming from a left-wing, liberal journalist - unless you know from context that he was talking about the 1880s, not the 1980s. He was not wishing the Kaiserreich back, he was just comparing different historical phases of it. Nevertheless, without the context, the quote would look highly strange, even if it is technically not a fake.

dwighttk 5y ago

They were isolating the various aspects one might use to detect fakes. Audio only, video only, political content only.