5 ms·

Beyond geopolitical concerns - as the tech becomes more widespread and easier to use, to the point where teenagers start using it to punk one another, no one wi

by parksy 7y ago

Beyond geopolitical concerns - as the tech becomes more widespread and easier to use, to the point where teenagers start using it to punk one another, no one will believe anything they see, hear or read ever again without an extremely robust digital provenance system (and who built that system? are they to be trusted? does it rely on intrusive measures? etc) It's great the developers and researchers are taking pause - this is great tech that could be used positively and creatively in so many areas - and with any new tech comes pain from misuse. If history is anything to go by, legislation is likely catch up slightly too late to stop the cultural impact, be overly broad, drag in a lot of relatively harmless outliers, undermine the positive use-cases and won't deter dedicated high-stakes bad actors.

pgt 7y ago
Personal SSL certificates? I don't see why original media can't be signed by the creator. We'll see similar measures as we do for entering passwords into non-SSL web pages: "Warning: this video is unsigned and claims to originate from NYTimes.com."
- faissaloo 7y ago
  Because that still relies on trust.
  - FungalRaincloud 7y ago
    Trust, to me, is not the problem. You can build trust. Known-good certificates can be distributed physically, and require signed messages for replacement. Or, we can develop schemes for distribution digitally via validated channels. For example, each worker at a company has a particular known-good digital presence, verified by their own public key, and distribution happens with them as the source, essentially creating an expanding ring of trust to the key being distributed. Violating such a ring of trust is not going to be easy, if it is well enough built. There are two issues I do see, though, and they're kind of the same issue. Right now, we have this concept of a central store of public certificates. It makes it easy for you to get a certificate for a particular entity, but it also makes the central store a target. If you can compromise a central store (or a machine that is attempting to access said central store), you probably have the resources to at least redirect the user to your own site and leave them none-the-wiser, and you probably have the resources to man-in-the-middle their connection entirely and just snoop your heart out. So central stores of trust are a bit of an issue, and the ways around that are non-trivial to set up. A good example is probably KeyBase, who allow you to certify your various online presences with your private key. So if someone wants to replace your information on KeyBase with their own one, and they have the resources to do so, now they also have to compromise all the places you've distributed that key to. Or, they have to compromise one of those centralized stores of trust.... The big issue with centralized stores of trust is that they build blind trust. That's the big issue with humans in general, though. We don't want to question what we're watching. And we probably don't want to be bothered with validating that the "trusted source" of the certificate used to sign this content is actually _trusted_. It's just too much mental overhead. We want it to be automatic. We want central stores of trust, because it's just _easier_. The work is going to be convincing people that _easier_ is dangerous, in this case. Or, it's going to be to convince software companies to build in inconvenient technology and not make it trivial to turn off.
    - intended 7y ago
      “Easier” is the whole point of the society you live in. To be fair, the point of society is trust. It’s a way to trust information and ensure the species is safe. The whole point of using markets and capitalism is because they generate more trust worthy results than top down driven systems. Until this mess, which makes it seem like a central authority will be better than a system that leaves nodes on the three open for manipulation. Essentially, we had a distributed decision making society. Now we found a hack to break that society structure. The cost for such a society to manage verification is absurdly high - every person will have to spend non trivial effort to verify that they are not being manipulated. In contrast central decision making societies like China will just avoid that cost and be more competitive, beating out western democratic systems.
- maxerickson 7y ago
  To mean anything it has to be signed by robustly secure hardware, with a manufacturer key.
- bin0 7y ago
  This is a good idea in theory, but I want you to think about how much difficulty we have with passwords all ready. Not confident this is workable.
ryandrake 7y ago
> as the tech becomes more widespread and easier to use, to the point where teenagers start using it to punk one another, no one will believe anything they see, hear or read ever again without an extremely robust digital provenance system People still believe grandma's easily Snope-able hoax E-mail forwards. We've had Photoshop for decades, yet people believe (and share/spread) doctored photos likely more today than ever. Pros can already make convincing doctored videos. I guarantee, in the 2024 election (maybe even as early as the 2020 one), we'll have professional-quality, flawless "video evidence" of one or more candidates saying or doing things they never said or did, produced by amateurs. Despite knowledge that this technology exists, most people will fall for it.
jacquesm 7y ago
> no one will believe anything they see, hear or read Yes they will. That's the one part, which is why there is a problem here in the first place. The second part is that people will stop believing media, which might actually be a good thing. The real problem with deepfakes is when the latency to modify something goes from hours to minutes to real-time, and even a live stream can no longer be trusted to be genuine. We're pretty close to that point, and for some use cases have passed it already.
- JKCalhoun 7y ago
  > The second part is that people will stop believing media, which might actually be a good thing. Why is that a good thing?
  - autokad 7y ago
    I dont know if it will be a good thing or not, but one possible consequence is that people will be less reactionary to videos, and thus videos that are taken out of context will do less harm.
  - jacquesm 7y ago
    I said it might be, not that it is. Besides that, people are just too plain gullible when it comes to media. It used to be said that if it is printed it must be true, now it is 'if it is on the internet, it must be true' leading to vast numbers of people being critically mis-informed about important issues.
- Faark 7y ago
  > The second part is that people will stop believing media, which might actually be a good thing. More likely: They won't trust media not in line with what they are already thinking. This won't help reduce confirmation bias. Thus already existing problems will likely be aggravated even more.
  - deleted 7y ago
    [deleted]
dclowd9901 7y ago
Real truth, I'm afraid, is disappearing, and there's no way to get it back. It's funny: We've always seen the eventual abilities of computer processed images (their eventually being indistinguishable from reality) as an absolute endpoint to the progress made in hardware and software over the years, but this threat of deep fakes seems to have surprised the lot of us.
- 9wzYQbTYsAIc 7y ago
  *has disappeared
celim307 7y ago
History itself will be a very interesting field to follow post deep-fake.
empath75 7y ago
You can’t believe everything you read right now and haven’t been able to since writing was invented. It’s only a relatively recent situation that we’ve had photography and audio recordings that were difficult to fake.

by parksy 7y ago

pgt 7y ago

Personal SSL certificates? I don't see why original media can't be signed by the creator. We'll see similar measures as we do for entering passwords into non-SSL web pages: "Warning: this video is unsigned and claims to originate from NYTimes.com."

faissaloo 7y ago
Because that still relies on trust.
- FungalRaincloud 7y ago
  Trust, to me, is not the problem. You can build trust. Known-good certificates can be distributed physically, and require signed messages for replacement. Or, we can develop schemes for distribution digitally via validated channels. For example, each worker at a company has a particular known-good digital presence, verified by their own public key, and distribution happens with them as the source, essentially creating an expanding ring of trust to the key being distributed. Violating such a ring of trust is not going to be easy, if it is well enough built. There are two issues I do see, though, and they're kind of the same issue. Right now, we have this concept of a central store of public certificates. It makes it easy for you to get a certificate for a particular entity, but it also makes the central store a target. If you can compromise a central store (or a machine that is attempting to access said central store), you probably have the resources to at least redirect the user to your own site and leave them none-the-wiser, and you probably have the resources to man-in-the-middle their connection entirely and just snoop your heart out. So central stores of trust are a bit of an issue, and the ways around that are non-trivial to set up. A good example is probably KeyBase, who allow you to certify your various online presences with your private key. So if someone wants to replace your information on KeyBase with their own one, and they have the resources to do so, now they also have to compromise all the places you've distributed that key to. Or, they have to compromise one of those centralized stores of trust.... The big issue with centralized stores of trust is that they build blind trust. That's the big issue with humans in general, though. We don't want to question what we're watching. And we probably don't want to be bothered with validating that the "trusted source" of the certificate used to sign this content is actually _trusted_. It's just too much mental overhead. We want it to be automatic. We want central stores of trust, because it's just _easier_. The work is going to be convincing people that _easier_ is dangerous, in this case. Or, it's going to be to convince software companies to build in inconvenient technology and not make it trivial to turn off.
  - intended 7y ago
    “Easier” is the whole point of the society you live in. To be fair, the point of society is trust. It’s a way to trust information and ensure the species is safe. The whole point of using markets and capitalism is because they generate more trust worthy results than top down driven systems. Until this mess, which makes it seem like a central authority will be better than a system that leaves nodes on the three open for manipulation. Essentially, we had a distributed decision making society. Now we found a hack to break that society structure. The cost for such a society to manage verification is absurdly high - every person will have to spend non trivial effort to verify that they are not being manipulated. In contrast central decision making societies like China will just avoid that cost and be more competitive, beating out western democratic systems.
maxerickson 7y ago
To mean anything it has to be signed by robustly secure hardware, with a manufacturer key.
bin0 7y ago
This is a good idea in theory, but I want you to think about how much difficulty we have with passwords all ready. Not confident this is workable.

ryandrake 7y ago

> as the tech becomes more widespread and easier to use, to the point where teenagers start using it to punk one another, no one will believe anything they see, hear or read ever again without an extremely robust digital provenance system People still believe grandma's easily Snope-able hoax E-mail forwards. We've had Photoshop for decades, yet people believe (and share/spread) doctored photos likely more today than ever. Pros can already make convincing doctored videos. I guarantee, in the 2024 election (maybe even as early as the 2020 one), we'll have professional-quality, flawless "video evidence" of one or more candidates saying or doing things they never said or did, produced by amateurs. Despite knowledge that this technology exists, most people will fall for it.

jacquesm 7y ago

> no one will believe anything they see, hear or read Yes they will. That's the one part, which is why there is a problem here in the first place. The second part is that people will stop believing media, which might actually be a good thing. The real problem with deepfakes is when the latency to modify something goes from hours to minutes to real-time, and even a live stream can no longer be trusted to be genuine. We're pretty close to that point, and for some use cases have passed it already.

JKCalhoun 7y ago
> The second part is that people will stop believing media, which might actually be a good thing. Why is that a good thing?
- autokad 7y ago
  I dont know if it will be a good thing or not, but one possible consequence is that people will be less reactionary to videos, and thus videos that are taken out of context will do less harm.
- jacquesm 7y ago
  I said it might be, not that it is. Besides that, people are just too plain gullible when it comes to media. It used to be said that if it is printed it must be true, now it is 'if it is on the internet, it must be true' leading to vast numbers of people being critically mis-informed about important issues.
Faark 7y ago
> The second part is that people will stop believing media, which might actually be a good thing. More likely: They won't trust media not in line with what they are already thinking. This won't help reduce confirmation bias. Thus already existing problems will likely be aggravated even more.
- deleted 7y ago
  [deleted]

dclowd9901 7y ago

Real truth, I'm afraid, is disappearing, and there's no way to get it back. It's funny: We've always seen the eventual abilities of computer processed images (their eventually being indistinguishable from reality) as an absolute endpoint to the progress made in hardware and software over the years, but this threat of deep fakes seems to have surprised the lot of us.

9wzYQbTYsAIc 7y ago
*has disappeared

celim307 7y ago

History itself will be a very interesting field to follow post deep-fake.

empath75 7y ago

You can’t believe everything you read right now and haven’t been able to since writing was invented. It’s only a relatively recent situation that we’ve had photography and audio recordings that were difficult to fake.