7 ms·

Before anyone is panicking, please remind yourself that whatever medium was used by humans as a form of communication in the last few thousand years, was also u

by lqet 7y ago

Before anyone is panicking, please remind yourself that whatever medium was used by humans as a form of communication in the last few thousand years, was also used for fakes. You only need a half-decent voice actor to pretend you are someone else on the phone. Fake photographs are as old as photography. Even the probably most immediate form of interpersonal communication, seeing another person, can be faked, which is demonstrated by the many Doppelgängers used by politicians in the past. The power of the house Habsburg basically had its roots in several imperial documents and bulls faked by Rudolf IV. in 1358 (which was only discovered in the 19th century). This isn't a new phenomenon that now means that we cannot trust anything anymore - it's only that the fakes are finally catching up with live video.

prox 7y ago
One rule there was in old times you always had 2 witnesses with you for any important event to vouch for. Reasoning was that it is a lot more difficult to do any trickery when more people are involved. Just because we scaled a lot of services in a digital network means our trust services need to adapt as well.
numlock86 7y ago
> You only need a half-decent voice actor to pretend you are someone else on the phone. You can even digitalize this part already. All you need are 5 seconds of recording and a pre-trained model in the desired language/accent. https://www.youtube.com/watch?v=0sR1rU3gLzQ https://www.youtube.com/watch?v=0sR1rU3gLzQ
- saurik 7y ago
  Right... but the point is that you _don't_ need fancy neural networks, you _only_ really need someone with an actually not that uncommon skill.
  - numlock86 7y ago
    Yes. But I think there is another way to look at this: You _don't_ need fancy people with a not so uncommon skill, you _only_ need a bit of computing power. Point is: I totally agree! The entry level to fake something is really low. No matter the technical value or skills you bring to the table.
    - saurik 7y ago
      But the point of the comment being responded to--which I also think is a very important point that it feels like you missed--is this isn't some new issue, caused by computers. Zeitgeist: panic, "the concept of truth is being destroyed by advancements in computers!!!" OP: sigh, "no... you have always been able to trick people, such as with audio, using a trained voice actor" You: "but, even with audio, you can also trick people with fancy new computers!" Me: ?!?
      - feanaro 7y ago
        The OP seems as if he understood that, but his argument is that this new technology makes the costs significantly lower. The lowering of cost has a strong impact in itself. In other words, almost no one would have bothered seeking out and then paying a talented individual with a somewhat uncommon skill in the vast majority of cases. Once this kind of software is available for free and packaged in such a package that almost zero effort is needed in order to achieve the same effect, a significantly higher number of people will do it. So something important has changed. I agree that the alarmist stance is still unwarranted, though.
  - _Microft 7y ago
    Using voice actors and doppelgängers does not scale. It is similar to the argument that automated surveillance is not bad because you could as well observe someone yourself. Sure that is possible but in reality it is limited to the niche of police and private detectors because it does not scale well.
- scarejunba 7y ago
  Very cool. Thanks for sharing. I wonder why the warbling is easier to detect in the male voices. They sound like a computer. But very cool advancement.
mikorym 7y ago
I'm glad to see this point being made fairly often. I see the "base reality" debate in a similar situation. Base reality could be the same thing as a religion, where the "god" is the system operator [1]. That way, you are "only" as scared of multiple levels of reality as you are of religion. But perhaps more importantly, that way you have 2000+ years of debate and reflection on the topic. [1] Not my idea.
philbarr 7y ago
Reminds me of this story from WW2 [0]. I'm sure I saw a documentary about this but that's the only reference I can find. "In late December 1942 Grimson and Allan Morris walked out of the camp with a group of German servicemen but were recaptured 2 days later, when they showed more courtesy than was expected from a German Officer. The prisoner forgery department had produced the documents necessary for the escape and the German uniforms to go over their civilian clothing. Both Grimson and Morris had again noticed that they had near doubles serving as officers in the German guard company who they impersonated and both had lain in wait for 3 nights fully disguised in their German uniforms waiting for the right moment when the two clones attended the show. Outside the wire they changed into civilian clothes disguised as foreign workers, walking to Sagan station and catching a train to the outskirts of Leipzig before being caught." [0] https://en.wikipedia.org/wiki/George_Grimson https://en.wikipedia.org/wiki/George_Grimson
ramraj07 7y ago
In the past, you needed to put effort to fake things. Of course, people still exist that believe the moonlanding videos were faked, but their hypothesis hinges on the government spending extraordinary amounts of money to produce the fake. Deepfakes are deeply troubling, because anyone with a computer (a requirement that's getting optional as well quite fast) can fake any video almost instantly. Consider just one sinister motive that's already making women in many countries cower in fear: if you reject a dude he can just make a deepfake porn video with you and release it as revenge. What's the equivalent of this in the distant past? All a guy could have done is "allege" the woman is so-and-so, which to be sure, probably did ruin some women's lives, but it's almost as if our society has devolved back to that same Stone age at best then. And we live in a time when "facts" already don't matter much anymore and people are just looking to find evidence that supports their claim alone rather than the truth. To not take this technology seriously, to brush it off because "sensible" people will still pursue to seek the truth, is the kind of smug arrogance about questionable tech that has led us to the current overall shit:sandwich state of affairs to begin with. Let's hope we learn some lessons at least.
- keiferski 7y ago
  What the original commenter is saying, and what I will agree will happen, is that the behavior of automatically trusting video as evidence will go away. I think this is actually a good thing if it encourages people to dig deeper into the sources of information. > If you reject a dude he can just make a deepfake porn video with you and release it as revenge. This will be troubling at first, but again in the long run it will likely just lead to people not taking these sort of videos seriously and interpreting them as a joke / weird stuff on the Internet. This already essentially happens with Photoshop and celebrities. > And we live in a time when "facts" already don't matter much anymore and people are just looking to find evidence that supports their claim alone rather than the truth This is how human beings have been since the beginning of time. There is no "Golden Age of Rationality" that we are exiting.
  - BiteCode_dev 7y ago
    > What the original commenter is saying, and what I will agree will happen, is that the behavior of automatically trusting video as evidence will go away. This is doubtful. People still believe in pictures despite it's so easy to forge. People believe obvious oral lies. Hell, people believe tweets. The video medium is incredibly powerful, and the damage it can do is huge. We are going to suffer for this, and most people won't even understand why.
keiferski 7y ago
Similar things happened in Russia and the Ottoman Empire - people claiming to be so-and-so, heir to the throne. As with anything else, if you are/aren't validated by society + the "gatekeepers of power", it doesn't really matter how good/bad your fake evidence is. https://en.wikipedia.org/wiki/False_Dmitry https://en.wikipedia.org/wiki/False_Dmitry
giarc 7y ago
In 1358 you could fake your identity and communicate with a handful of people. In 2020 you can fake your identity and communicate with 7.5 billion people at once. I think there is a difference.

by lqet 7y ago

prox 7y ago

One rule there was in old times you always had 2 witnesses with you for any important event to vouch for. Reasoning was that it is a lot more difficult to do any trickery when more people are involved. Just because we scaled a lot of services in a digital network means our trust services need to adapt as well.

numlock86 7y ago

> You only need a half-decent voice actor to pretend you are someone else on the phone. You can even digitalize this part already. All you need are 5 seconds of recording and a pre-trained model in the desired language/accent. https://www.youtube.com/watch?v=0sR1rU3gLzQ https://www.youtube.com/watch?v=0sR1rU3gLzQ

saurik 7y ago
Right... but the point is that you _don't_ need fancy neural networks, you _only_ really need someone with an actually not that uncommon skill.
- numlock86 7y ago
  Yes. But I think there is another way to look at this: You _don't_ need fancy people with a not so uncommon skill, you _only_ need a bit of computing power. Point is: I totally agree! The entry level to fake something is really low. No matter the technical value or skills you bring to the table.
  - saurik 7y ago
    But the point of the comment being responded to--which I also think is a very important point that it feels like you missed--is this isn't some new issue, caused by computers. Zeitgeist: panic, "the concept of truth is being destroyed by advancements in computers!!!" OP: sigh, "no... you have always been able to trick people, such as with audio, using a trained voice actor" You: "but, even with audio, you can also trick people with fancy new computers!" Me: ?!?
    - feanaro 7y ago
      The OP seems as if he understood that, but his argument is that this new technology makes the costs significantly lower. The lowering of cost has a strong impact in itself. In other words, almost no one would have bothered seeking out and then paying a talented individual with a somewhat uncommon skill in the vast majority of cases. Once this kind of software is available for free and packaged in such a package that almost zero effort is needed in order to achieve the same effect, a significantly higher number of people will do it. So something important has changed. I agree that the alarmist stance is still unwarranted, though.
- _Microft 7y ago
  Using voice actors and doppelgängers does not scale. It is similar to the argument that automated surveillance is not bad because you could as well observe someone yourself. Sure that is possible but in reality it is limited to the niche of police and private detectors because it does not scale well.
scarejunba 7y ago
Very cool. Thanks for sharing. I wonder why the warbling is easier to detect in the male voices. They sound like a computer. But very cool advancement.

mikorym 7y ago

I'm glad to see this point being made fairly often. I see the "base reality" debate in a similar situation. Base reality could be the same thing as a religion, where the "god" is the system operator [1]. That way, you are "only" as scared of multiple levels of reality as you are of religion. But perhaps more importantly, that way you have 2000+ years of debate and reflection on the topic. [1] Not my idea.

philbarr 7y ago

Reminds me of this story from WW2 [0]. I'm sure I saw a documentary about this but that's the only reference I can find. "In late December 1942 Grimson and Allan Morris walked out of the camp with a group of German servicemen but were recaptured 2 days later, when they showed more courtesy than was expected from a German Officer. The prisoner forgery department had produced the documents necessary for the escape and the German uniforms to go over their civilian clothing. Both Grimson and Morris had again noticed that they had near doubles serving as officers in the German guard company who they impersonated and both had lain in wait for 3 nights fully disguised in their German uniforms waiting for the right moment when the two clones attended the show. Outside the wire they changed into civilian clothes disguised as foreign workers, walking to Sagan station and catching a train to the outskirts of Leipzig before being caught." [0] https://en.wikipedia.org/wiki/George_Grimson https://en.wikipedia.org/wiki/George_Grimson

ramraj07 7y ago

In the past, you needed to put effort to fake things. Of course, people still exist that believe the moonlanding videos were faked, but their hypothesis hinges on the government spending extraordinary amounts of money to produce the fake. Deepfakes are deeply troubling, because anyone with a computer (a requirement that's getting optional as well quite fast) can fake any video almost instantly. Consider just one sinister motive that's already making women in many countries cower in fear: if you reject a dude he can just make a deepfake porn video with you and release it as revenge. What's the equivalent of this in the distant past? All a guy could have done is "allege" the woman is so-and-so, which to be sure, probably did ruin some women's lives, but it's almost as if our society has devolved back to that same Stone age at best then. And we live in a time when "facts" already don't matter much anymore and people are just looking to find evidence that supports their claim alone rather than the truth. To not take this technology seriously, to brush it off because "sensible" people will still pursue to seek the truth, is the kind of smug arrogance about questionable tech that has led us to the current overall shit:sandwich state of affairs to begin with. Let's hope we learn some lessons at least.

keiferski 7y ago
What the original commenter is saying, and what I will agree will happen, is that the behavior of automatically trusting video as evidence will go away. I think this is actually a good thing if it encourages people to dig deeper into the sources of information. > If you reject a dude he can just make a deepfake porn video with you and release it as revenge. This will be troubling at first, but again in the long run it will likely just lead to people not taking these sort of videos seriously and interpreting them as a joke / weird stuff on the Internet. This already essentially happens with Photoshop and celebrities. > And we live in a time when "facts" already don't matter much anymore and people are just looking to find evidence that supports their claim alone rather than the truth This is how human beings have been since the beginning of time. There is no "Golden Age of Rationality" that we are exiting.
- BiteCode_dev 7y ago
  > What the original commenter is saying, and what I will agree will happen, is that the behavior of automatically trusting video as evidence will go away. This is doubtful. People still believe in pictures despite it's so easy to forge. People believe obvious oral lies. Hell, people believe tweets. The video medium is incredibly powerful, and the damage it can do is huge. We are going to suffer for this, and most people won't even understand why.

keiferski 7y ago

Similar things happened in Russia and the Ottoman Empire - people claiming to be so-and-so, heir to the throne. As with anything else, if you are/aren't validated by society + the "gatekeepers of power", it doesn't really matter how good/bad your fake evidence is. https://en.wikipedia.org/wiki/False_Dmitry https://en.wikipedia.org/wiki/False_Dmitry

giarc 7y ago

In 1358 you could fake your identity and communicate with a handful of people. In 2020 you can fake your identity and communicate with 7.5 billion people at once. I think there is a difference.