8 ms·

Oh that's great news \s Do other people read these types of headings and just think immediately about how freaking scary this stuff is going to get...if it's a

• sendfoods 4y ago
  Yeah, I am increasingly nervous about some point in the future where you cannot trust any digital medium because of deep fakes, NERFs, voice cloning... The only possible solution I have seen being talked about is digitally signing any and all "real" content, similar to GPG/SSH.
  • ravivyas 4y ago
    Photoshop and minor video manipulation is enough to trick people into believing fake news, this will make it harder. I see political parties doing simple edits to make people believe anything, this will push it multiple notches higher.
    • jacquesm 4y ago
      That they can do so without consequences is almost as scary as that they can do so in the first place.
      • willnonya 4y ago
        Fraud, slander etc are already illegal. There are consequences for crossing these lines.
        • jacquesm 4y ago
          AFAIK nothing ever happened to the political parties that use these tactics. But I'll be happy to be corrected.
        • pjc50 4y ago
          Not in politics they aren't. There was an incredibly high bar to clear for the Alex Jones libel lawsuit to succeed.
    • teekert 4y ago
      You really have low exceptions for humanity. Most people are not complete idiots you know. I think your opinion is pretty elitist. Or do you think you yourself will fall for this as well?
      • ravivyas 4y ago
        Less elitist(might really sound like one), more realist who knows that masses devolve to the lowest common denominator of sorts based on the information they are provided
      • citizenkeen 4y ago
        Not the person you’re responding to, but I don’t think it’s egotist at all. I’m sure I’ll fall for some news manipulation. I’m sure I already have. I don’t think it’s elitist to think many people will be duped. I think it’s arrogant to think you won’t.
      • pjc50 4y ago
        Bunch of people just stormed the Brazilian government buildings yesterday over a fake "election theft" narrative. This stuff will be constantly tuned until enough people do fall for it, like advertising. There's also the unravelling biography of https://en.wikipedia.org/wiki/George_Santos#Scandals https://en.wikipedia.org/wiki/George_Santos#Scandals (you know someone's doing well when their "scandals" section has ten subheadings! And that's all old-fashioned manual fraud) (I'm now wondering how far you could get with an elected official who's entirely deepfaked and only appears in recordings. Probably all the way to being sworn in, at which point someone has to appear in person. People have occasionally been elected despite being dead, so it's not impossible)
        • swader999 4y ago
          Or millions of people passively accepted the results of a stolen election. That would be an equally viable scenario with this technology. You just won't know the truth anymore.
          • 4y ago
    • iso1631 4y ago
      > Photoshop and minor video manipulation is enough to trick people into believing fake news If the news confirms what they think is the truth, it doesn't need manipulation, just a fake heading https://www.snopes.com/fact-check/climate-protest-ukraine-body-bag/ https://www.snopes.com/fact-check/climate-protest-ukraine-bo... for example (and in that case it even had the true caption on the video, just not in English)
  • hef19898 4y ago
    I don't believe anything digital anymore for quite a while. Heck, pictures have been staged and faked since the beginning of photography. Modern tech just makes it easier. Maybe, just maybe, human society will move back to personal interaction, since apparently you cannot trust call, voice and video, anymore in the future.
    • pjc50 4y ago
      > human society will move back to personal interaction Which implies a much narrower, less diverse, more elitist system of both politics and economy, as it gets harder to trust people from outside your existing circle.
      • hef19898 4y ago
        True. Sometimes it seems some people read every CyberPunk story and considered it an instruction instead of something between a parody and a warning.
      • brightstep 4y ago
        Hard to get more elitist than our politics and economy are now at the height of digital interconnection. The liberal democratic revolutions and popular democratic swell of the US labor movement under FDR all happened without the internet. A return of social connection to the community would be very much to our political benefit.
• chmod775 4y ago
  At some point natural people are going to use cryptographic signatures just to authenticate themselves in daily life. No video, audio, or text can be presumed real unless signed by the human subject.
  • zigman1 4y ago
    But... That's good no? > No video, audio, or text can be presumed real unless signed by the human subject. Where is the controversy in that?
    • rippercushions 4y ago
      It would be nice if we were at 100% signing, but currently we're at ~0%, and the transition is going to be rough.
      • willnonya 4y ago
        Why would that be nice? Follow the thou g ht through to how it would work and what it would mean in terms of privacy for the average person. Your niceties seem dystopian to me.
    • jddj 4y ago
      Agreed. The time between now and then is much scarier. Companies can't even wrap their heads around sim swapping scams and social engineering of secret questions.
    • Someone 4y ago
      Those subjects could lie. Journalists would have a hard time showing that an interviewee actually said what claim they said. Wire-tapping by the police also would become useless in court. ⇒ I would adjust it to “unless it has a reliably chain of trust”. That would mean that people could judge differently about whether to accept some things as true, but I think that’s unavoidable.
  • TeMPOraL 4y ago
    To do that, any meaningful communication will have to be recorded to be signed by the person, on a device provided to them by their Trust Authority company. Can't be just any device, because the chain of custody is as strong as its weakest link. Depending on the country, the Trust Authorities will be private companies with large regulatory exposure, like banks, or just governments themselves. But hey, at least this will solve the problem of journalists quoting people out of context: they won't be able to claim you've said something without you personally signing off on it with your cryptographic key.
• dsr_ 4y ago
  Yes. Then I listened to the samples, and was relieved that it's not good enough to handle a well-known voice impersonation... yet. My primary worry is for criminal misuse. There's a class of scams where the scammers find elderly victims, call them, and pretend to be their children or grandchildren on vacation in a plausible city and suddenly in need of money to help get their passport back or a ticket home. There are any number of cases where I have accepted an order worth large sums of money over the phone from someone I have previously dealt with and feel familiar with their voice.
  • rwmj 4y ago
    Only a year ago, my bank replaced their (already not very secure) second factor authentication with voice recognition.
    • toomuchtodo 4y ago
      It’s very concerning how ubiquitous this very soon to be insecure authentication factor is. https://www.fidelity.com/security/fidelity-myvoice/overview https://www.fidelity.com/security/fidelity-myvoice/overview https://www.schwab.com/voice-id https://www.schwab.com/voice-id https://blogs.perficient.com/2016/04/18/3-benefits-of-vanguards-voice-verification-feature/ https://blogs.perficient.com/2016/04/18/3-benefits-of-vangua... https://www.chase.com/personal/voice-biometrics https://www.chase.com/personal/voice-biometrics
    • chinathrow 4y ago
      They will recycle it for sure once fraud hits their internal thresholds. Until that: self defense.
      • SketchySeaBeast 4y ago
        It's mind boggling that the banks, who you think would have the most to gain from staying ahead of the tech scam curve, can't be bothered and instead continue to play a slow defense. I suppose they aren't losing THEIR money.
  • 4y ago
• jacquesm 4y ago
  "Hi mom, it's Chrissy, I am calling you from someone else's phone, my phone got stolen. Do you have a pen handy, I'll wait. Ok, here is my new number, take the old one off your contacts right away, surely some scammer has it, and can you forward me 1000 bucks or so for a new phone, I'll pay it back when I have the new phone set up, here is my account number...." When it's a text message it's likely going to ring a bell somewhere (hopefully an alarm bell), if it is a voice message and it sounds like the original many more people are going to fall for it.
  • teekert 4y ago
    So, we need to work on this ringing bells. Sure some people will fall for it initially, but then we adapt.
    • Gigachad 4y ago
      Scammers have been scamming since the beginning of society
  • sendfoods 4y ago
    My god...I haven't really thought about scamming at all, I was "just" worried about manipulation of media and politics. New nightmare unlocked.
    • hackerman123469 4y ago
      "Okay, let me just video call you because I also need to show you something" and then the scammer hung up and was to never be seen again.
      • probably_wrong 4y ago
        FYI, there's the following trick I read here some time ago: The scammer starts a video call with the person they want to impersonate and record it. When that person takes the call, the scammer doesn't say anything. This creates a 5-10 seconds video of the person looking at the camera waiting for the scammer to say something, until they get fed up and hung up. The scammer then calls the victim. They offer a video call as verification straight away, and they play that short video.
• dilawar 4y ago
  Recently in India, a politician deepfaked the voice to deliver his speech in multiple languages [1]. Can't comment on the quality of tech since i dont speak this language but it is pretty alarming to me. 2% vote swings make a huge difference in Indian politics since there are many regional parties. Almost more than 2/3rd of Indian voters didn't vote for modi yet his party is doing very well. [1] https://www.theverge.com/2020/2/18/21142782/india-politician-deepfakes-ai-elections https://www.theverge.com/2020/2/18/21142782/india-politician...
  • pjc50 4y ago
    Interesting. Providing a translation is an accessibility measure, but it seems that the "fake" here is matching his lip movements to the translation. Almost not worthy of calling it a fake since he (not somebody else) did say those things (in a different language), and it's endorsed by him. The unendorsed version is far more dangerous.
  • thalassophobia 4y ago
    I think this is one of the cases where there is nothing wrong with using deepfakes. The person mimicked gave explicit consent and they did not do any harm with the tool. If technology increases accessibility of true political choice then it's good technology. That said, I don't understand why the parent and the article paint this as a bad thing.
    • jwestbury 4y ago
      When language is tightly coupled to cultural identity, using this kind of fake is lying to voters, saying, "I'm one of you." It's a new use of technology to repeat an old lie. I think it's a bad thing, but it's hardly the most dangerous and novel application of this technology for evil.
      • IgorPartola 4y ago
        Had he used a translator, it would have been more obvious but possibly had the same effect (if what he had to say resonated with the voters). So what you are against is the idea that it’s not labeled as “virtual translator was used to produce this speech”?
• teekert 4y ago
  When everyone know this is possible, we need better methods to verify what is real. We'll handle it, people said the same about photoshop.
  • swarnie 4y ago
    I suspect it is already widely known. My employer is known for tracking a decade behind the curve and we already have financial processes in place to prevent this vector from being exploited. That leads me to believe the majority must already have as well.
  • _visgean 4y ago
    Photoshop isnt realtime. This will be.
• pjc50 4y ago
  People talk about cryptography, which has fairly terrible adoption rates outside of invisible systems like TLS, but in practice the way humans authenticate sources is heuristic: do I trust the person telling me this? Does it agree with things I already know? That is, you have a "filter bubble" which tells you that there isn't going to be a Nigerian prince with some money waiting for you (unknown sender, unlikely message). Reduced cost fakes force people to make their filter bubble smaller by raising the threshold for faking. More broadly, the question of "high trust vs. low trust" society.
• worldsavior 4y ago
  No because I know as technology advances, security advances.
• mrtksn 4y ago
  With all that ML stuff going on we can no longer trust our devices to guarantee authenticity of their behaviour. By that, I mean we can no longer be sure that the human voice produced by the speaker of our handheld computers is happening because a human somewhere else spoke to a microphone. Unfortunately the move towards it is nothing new, we gradually lost text and images and now the remaining media is closing. This will probably decimate online public forums completely and off-person communications will en up being P2P through trusted devices. Closed communities in WhatsApp, Telegram, Slack and Discord already replaced forums. Maybe if infestation of closed communities with machines becomes a thing, maybe the real life gatherings will make a comeback? I don't know but I used to mock Sci-Fi movies about portraying aliens as uncivilised that don't seem to have the means to have developed all that tech but I'm no longer sure. Maybe what will happen is, we will create a symbiotic life where machine will no longer be tools but partners and they will take care of us to reduce us to our basic instinct.
• tootie 4y ago
  A company called Lyrebird claimed they could replicate a voice in less time and had demos available a few years ago. The deepfake scare of 2019 also seems like a long time ago.
• lannisterstark 4y ago
  Oh great, more fearmongering. What cases do you think it will be used wrongly in? It can't do 1:1 realistic phone calls, the processing it'd need for that is beyond the reach of any average user. I just would like this so I can copy-paste books/text and listen to it in the voices of my favorite narrators, but can't have that, can we? No, it'll be super dangerous, I'll listen myself to nuking a country or some such.
  • r-zip 4y ago
    If you seriously don’t think this will be widely abused, then you’re extremely optimistic. Spreading propaganda is the most obvious use case, assuming the resource requirement remains high. I don’t think it’s far-fetched either. If these models become less resource-hungry, then we’re in for much worse.
  • Tao3300 4y ago
    > the voices of my favorite narrators If you really like them and their work, why would you want a forgery?
  • alfalfasprout 4y ago
    Compute keeps getting cheaper. This will absolutely be within the reach of a dedicated user in a very short span of time. If you can't see the danger in this, perhaps it's more that you're ok with it.
• XorNot 4y ago
  No, I read these types of headings and think how awesome open world gaming is going to be when any random developer can fully voice their game cheaply. The AAA benefits too: you get a lot of stilted dialogue when the script and gameplay diverge and you have to work with what you have - a world where the writers can just redo that on the fly pretty much up till release would mean we'd be looking at a new era of dynamic storytelling. It'll be interesting when someone releases the first MMO which combines GPT models and VO generation to fill out the in-universe world with dynamic characters who can react to surrounding events.