4 ms·

That's super cool. I am worried about the potential abuse of this service, are there any existing services that can help to identify audio deep fakes like this

That's super cool. I am worried about the potential abuse of this service, are there any existing services that can help to identify audio deep fakes like this one is for making them? Found Resemblyzer: https://github.com/resemble-ai/Resemblyzer https://github.com/resemble-ai/Resemblyzer

GrantZvolsky 6y ago
Such a system will always suffer from false positives and false negatives. On a more positive note, when deepfakes become a problem, we will see the emergence of a culture where unsigned authoritative content is not paid any attention.
- kibwen 6y ago
  > we will see the emergence of a culture where unsigned authoritative content is not paid any attention If current events are any indication, that culture will only emerge 30 years after the tech becomes widely usable, and in the interim will lead to absolute chaos in the form of weaponized disinformation.
  - totetsu 6y ago
    It could be built into Youtube or twitter to pop up a little "this could be a fake voice" banner
- mywittyname 6y ago
  This is a big issue! Lots of bad things happen, and they are only surfaced because the person in question didn't notice the surreptitious recording. When deep fakes becomes a problem, it will give these people plausible deniability and they can just reject it as "fake news."
  - mycall 6y ago
    Then someone needs to get ahead of this and either (1) detect these deep fake recordings or (2) create trusted channels that avoids them.
- underwater 6y ago
  Photoshop has existed for a long time, but people still take photos at face value.
  - mhh__ 6y ago
    Photos were being doctored long before then, even. For example, https://en.wikipedia.org/wiki/Censorship_of_images_in_the_Soviet_Union#Censorship_of_historical_photographs https://en.wikipedia.org/wiki/Censorship_of_images_in_the_So...
    - Gibbon1 6y ago
      I'm reminded of a comment by someone that studied the old Eugenics movement. Mentioned finding that one of the major books on the subject the author had carefully edited photo's of 'degenerate people' to look even more crude an unintelligent. Also photographer friend of mine said; a great photographer doesn't need to Photoshop anything to lie to you.
      - GrantZvolsky 6y ago
        > Also photographer friend of mine said; a great photographer doesn't need to Photoshop anything to lie to you. I couldn't agree more if they used the word lie in the more general sense as a synonym of deception. The availability of fakes may not become a problem because the most effective deception doesn't involve telling untruths. To give an example, suppose Russia Today and Fox News report on the same event. There's a set of facts. RT picks a subset and reports it from their point of view. Fox picks another subset and presents their view. The resulting articles may give readers vastly different interpretation of the event and no untruths had to be involved.
azinman2 6y ago
I wonder if at some point we need to create legal requirements that all deepfakes have some kind of human-invisible (or visible?) fingerprint for identification, restriction on frequency range, etc. We have crypto export restrictions, why not put handcuffs on this as well which has the potential for probably larger scale harm?
- searchableguy 6y ago
  That will only stop law abiding citizens from committing crime.
  - azinman2 6y ago
    That’s a big start. If you look at Reddit, you’ll notice all these deepfake porns are using scripts/apps that people have packaged together. Those and any commercial variants can abide by such restrictions which will help document the fakeness once this goes mainstream. Only the super technical would be able to get around it, and if tutorials etc come out then you have legal grounds to go after it to minimize. Don’t let perfect be the enemy of good. This has potential to literally cause spilled blood, fraud, etc. Better to have it for some than for zero.
    - searchableguy 6y ago
      I don't agree with the solution unless we as a society stop putting trust on any digital media but at that point, this is not necessary. Many governments would love to use this tech so they have incentive to stop others from using it and still let people believe in digital evidence by putting half assed solution like the one you proposed. The cat is out of the bag. Digital media should not be trusted blindly.
      - azinman2 6y ago
        I don’t appreciate the condescension. It’s half assed in that I put a rough direction out there for conversation for a huge problem that will inevitably cause social problems. Your reply isn’t in line with HN guidelines and certainly doesn’t make me want to participate in conversation with you about important topics.

GrantZvolsky 6y ago

Such a system will always suffer from false positives and false negatives. On a more positive note, when deepfakes become a problem, we will see the emergence of a culture where unsigned authoritative content is not paid any attention.

kibwen 6y ago
> we will see the emergence of a culture where unsigned authoritative content is not paid any attention If current events are any indication, that culture will only emerge 30 years after the tech becomes widely usable, and in the interim will lead to absolute chaos in the form of weaponized disinformation.
- totetsu 6y ago
  It could be built into Youtube or twitter to pop up a little "this could be a fake voice" banner
mywittyname 6y ago
This is a big issue! Lots of bad things happen, and they are only surfaced because the person in question didn't notice the surreptitious recording. When deep fakes becomes a problem, it will give these people plausible deniability and they can just reject it as "fake news."
- mycall 6y ago
  Then someone needs to get ahead of this and either (1) detect these deep fake recordings or (2) create trusted channels that avoids them.
underwater 6y ago
Photoshop has existed for a long time, but people still take photos at face value.
- mhh__ 6y ago
  Photos were being doctored long before then, even. For example, https://en.wikipedia.org/wiki/Censorship_of_images_in_the_Soviet_Union#Censorship_of_historical_photographs https://en.wikipedia.org/wiki/Censorship_of_images_in_the_So...
  - Gibbon1 6y ago
    I'm reminded of a comment by someone that studied the old Eugenics movement. Mentioned finding that one of the major books on the subject the author had carefully edited photo's of 'degenerate people' to look even more crude an unintelligent. Also photographer friend of mine said; a great photographer doesn't need to Photoshop anything to lie to you.
    - GrantZvolsky 6y ago
      > Also photographer friend of mine said; a great photographer doesn't need to Photoshop anything to lie to you. I couldn't agree more if they used the word lie in the more general sense as a synonym of deception. The availability of fakes may not become a problem because the most effective deception doesn't involve telling untruths. To give an example, suppose Russia Today and Fox News report on the same event. There's a set of facts. RT picks a subset and reports it from their point of view. Fox picks another subset and presents their view. The resulting articles may give readers vastly different interpretation of the event and no untruths had to be involved.

azinman2 6y ago

I wonder if at some point we need to create legal requirements that all deepfakes have some kind of human-invisible (or visible?) fingerprint for identification, restriction on frequency range, etc. We have crypto export restrictions, why not put handcuffs on this as well which has the potential for probably larger scale harm?

searchableguy 6y ago
That will only stop law abiding citizens from committing crime.
- azinman2 6y ago
  That’s a big start. If you look at Reddit, you’ll notice all these deepfake porns are using scripts/apps that people have packaged together. Those and any commercial variants can abide by such restrictions which will help document the fakeness once this goes mainstream. Only the super technical would be able to get around it, and if tutorials etc come out then you have legal grounds to go after it to minimize. Don’t let perfect be the enemy of good. This has potential to literally cause spilled blood, fraud, etc. Better to have it for some than for zero.
  - searchableguy 6y ago
    I don't agree with the solution unless we as a society stop putting trust on any digital media but at that point, this is not necessary. Many governments would love to use this tech so they have incentive to stop others from using it and still let people believe in digital evidence by putting half assed solution like the one you proposed. The cat is out of the bag. Digital media should not be trusted blindly.
    - azinman2 6y ago
      I don’t appreciate the condescension. It’s half assed in that I put a rough direction out there for conversation for a huge problem that will inevitably cause social problems. Your reply isn’t in line with HN guidelines and certainly doesn’t make me want to participate in conversation with you about important topics.