6 ms·

These tools make it very easy to scam vulnerable people, and have pretty limited use otherwise.

• tsujamin 2y ago
  Bulldozing grandma is just the cost of technological progress /s
  • uh_uh 2y ago
    This tech is going to be ubiquitous, it's just too easy to distribute it. Grandma better starts adapting now.
    • thejazzman 2y ago
      Because people make it so, not because the natural order of the world gets us there For some reason because we can validates that we should. Any jackass has the power of a research team of phds. It's kinda weird.
      • uh_uh 2y ago
        Demanding responsible behaviour from everybody is not going to work. Some people don't care about negative externalities that much and it's enough if only a few of them decide not to play ball. So either grandma needs to adapt which will upset some people or distributing the tech should be regulated/prosecuted which will upset another group of people.
        • rockemsockem 2y ago
          I think either way grandma needs to adapt though since Russian scammers and trolls are still going to run scams with fake voices.
          • 123yawaworht456 2y ago
            how very politically correct of you to pretend it's Russians who scam your grandmas
            • rockemsockem 2y ago
              Insert any other country you like that doesn't have extradition agreements with the United States. Any other country the law can at least ostensibly be enforced there, even if it isn't always
      • chefandy 2y ago
        Indeed. Humans ascended to dominance because we can cooperate. This every-man-for-themself idea is an aberration, not the natural order as so many claim. It’s rather astounding to think otherwise considering the logistics of how we’re communicating right now.
        • uh_uh 2y ago
          Cooperation works if the potential damage caused by a rouge actor is sufficiently low. Otherwise, it's too easy to sabotage things. This is why we don't want random rouge states to have nukes. AI will give so much leverage to rouge actors that it will significantly shift the game theory in favour of not cooperating.
          • chefandy 2y ago
            > Cooperation works if the potential damage caused by a rouge actor is sufficiently low. Otherwise, it's too easy to sabotage things. This is why we don't want random rouge states to have nukes. AI will give so much leverage to rouge actors that it will significantly shift the game theory in favour of not cooperating. Governments successfully collectively controlling dangerous things so they don’t fall into the hands of rogue bad actors fundamentally opposes the extreme individualist every-man-for-himself perspective in every conceivable way. It’s the absolute opposite of “it’s everybody’s responsibility to protect themselves because everybody else is only going to look out for themselves.” And when individuals have that much leverage, collective action is the only conceivable way to oppose it. Some of those things might be cultural, like mores, some might be laws, some might be more martial. I don’t see how extreme individualism even theoretically could be more powerful.
            • uh_uh 2y ago
              Are you suggesting government action against putting up code like this to GitHub? It’s ok if you are, but I want to put into more concrete terms what we’re talking about.
              • 2y ago
    • chefandy 2y ago
      You can’t adapt around brain age making it more difficult to distinguish truth from lies.
    • casey2 2y ago
      Yeah, I don't really get the hulabaloo, if granny doesn't have the mental fortitude to keep up with the times she shouldn't be managing her own money. I guess better her son/daughter than a scammer but both are better than letting money rot. Put granny on foodstamps and pay $1 for her rent controled housing be done with it.
      • zelphirkalt 2y ago
        Are we forgetting, that there are many elderly people without living descendants?
  • weq 2y ago
    This tech is not only great for bulldozing grandma, its great at stealing content from other creators and rebranding it as your own. Based on the github, it kind of seems like thats exactly whats being advertised as the use case. Steal content from BBC, cut it up and pull the noise out/vocals/revoice the content so the algorithm cant detect the plagorism easily. The imagine detection is no where no the audio detection for copyright strikes. There is a massive problem with this on youtube. Pretty much every category on youtube now has a host of these bots trolling content and playing the youtube strike system like a banjo. There are channels detected to showing you how to setup these content mills. This tool can make you good money.
    • sfjailbird 2y ago
      First generative AI destroyed Google search, and now it has pretty much destroyed YouTube. Social platforms, including this one, are probably goners too. We live in interesting times.
• Larrikin 2y ago
  I'm absolutely using celebrity voices for my Home Assistant voice. Amazon has spent the last couple years removing the voices for Alexa that people had paid for.
  • nickthegreek 2y ago
    I’d love some more info on using custom voices in HA. I have an esp32-s3-box that I am setting up holiday to do voice with HA.
  • pmarreck 2y ago
    If you have a how—to, I’d love to work on one for my home. I feel like this is all right around the corner…
• chefandy 2y ago
  Gen AI space to everyone else: “Your computer scientists were so preoccupied with whether or not they should, they didn’t stop to think if they could just do it anyway”
• chefandy 2y ago
  To be fair, they’ve got pretty serious potential for letting tech companies get paid for a seasoned voice actor’s unique delivery, tone, inflection, etc rather than the voice actor themselves.
  • whaaaaat 2y ago
    > they’ve got pretty serious potential for letting tech companies get paid for a seasoned voice actor’s unique delivery, tone, inflection, etc rather than the voice actor themselves. I think you mean "steal the labor of an actor"?
    • chefandy 2y ago
      Sure, and people that already agree with you will feel good reading it, but other people who don’t agree see it as an attack. It’s pretty much impossible to slip a new idea into someone’s mind if your approach made them slam the door before even considering it. So what’s the benefit of saying it like that?
      • gmueckl 2y ago
        It calls attention to the ethical implications of using a part of someone else's personal identity without their direct involvement.
        • MrDrMcCoy 2y ago
          Indirect involvement can still be ok within the confines of a license agreement for using the actor's voice.
          • gmueckl 2y ago
            But this requires a legal framework that mandates such licenses and effective emforcement / procecution of violations. As far as I know, most countries are lagging behind when it comes to updating legislation to set binding rules around that.
• ranger_danger 2y ago
  How many victims will it take for lawmakers to do something about this?
  • tiborsaas 2y ago
    It's already illegal to scam somebody. While it's always positive to protect people more, what can be done here? Any alternative I can imagine is massively oppressive of the current state of the software industry. You can regulate large companies, you can regulate published software sold for profit, but it's impossible to regulate free and open source tools. You essentially have to regulate access to computing power if you want to prevent bad actors doing bad things using these sort of tools.
    • bryanrasmussen 2y ago
      >You can regulate large companies, you can regulate published software sold for profit, but it's impossible to regulate free and open source tools. Regulation is putting legal limitations on things, if it is impossible to regulate free and open source tools then it would be impossible to regulate murder and lots of other things, but it turns out it isn't impossible, sure - murder happens - but people get caught for it and punished. Sorry, but this argument is much like the early internet triumphalism - back when people said it was impossible to regulate. Turns out lots of countries now regulate it.
      • tiborsaas 2y ago
        It depends on what you do with the tool. Going with your murder analogy, if there's a stabbing epidemic what do you do? 1) Ban knives 2) invest in public safety 3) investigate the root causes and improve on them? I'm also not sure what's so regulated about the internet besides net neutrality in certain countries. Of course the government can put limits on the network, like banning services, but it's easy since they are rather easy to target. With content traveling on the network it's much harder to say if it's legit or not. > lots of countries What about those countries that don't regulate it and people will keep pumping out better, leaner and faster models from there? Spreading software is trivial, all you achieve is the public won't be aware of what's possible. The more I think about it if anything should be regulated that's a requirement to provide third party (probably government backed) ID verification system so it would be possible for my mom to know it's me calling here. Basically kill called ID spoofing.
• rockemsockem 2y ago
  Quit being a doomer or keep it to yourself. This reminds me of the sound boards that were popular in the early 2000s except way more versatile. Some things are just good for people to have fun and THAT'S OKAY.
  • whaaaaat 2y ago
    People are allowed to recognize the realistic negative outcomes of technology, especially on a forum that frequently discusses the tradeoffs of modern, cutting edge technologies.
    • rockemsockem 2y ago
      So many AI posts are overrun with this kind of complaining from folks with limited imaginations. On a forum that frequently discusses technology with enthusiasm you'd think there'd be more enthusiasm and more constructive criticism instead of blanket write-offs.
      • Mordisquitos 2y ago
        I would argue that being able to see the drawbacks and potential negative externalities of a new technology is not a sign of a "limited imagination", but quite the contrary. An actual display of a limited imagination is the inability to imagine how a new technology can (and will) be abused in society by bad actors.
        • Ukv 2y ago
          Developing some insight on its negative potential could demonstrate imagination, but the claim that it could be used to scam people is pretty much just rote repetition by now - an obligatory point made in every article and under every post about this tech (and not something that I think actually works out in practice the way most imagine it, since cold-call scam operations that dial numbers at a huge scale expecting most not to pick up can't really find a voice clip prior to each automated call). As for positive applications, some I see: * Allowing those with speech impairments to communicate using their natural voice again * Allowing those uncomfortable with their natural voice, such as transgender people, to communicate closer to how they wish to be perceived * Translation of a user's voice, maintaining emotion and intonation, for natural cross-language communication on calls * Professional-quality audio from cheap microphone setups (for video tutorials, indie games, etc.) * Doing character voices for a D&D session, audiobook, etc. * Customization of voice assistants, such as to use a native accent/dialect * Movies, podcasts, audiobooks, news broadcasts, etc. made available in a huge range of languages * If integrated with something like airpods, babelfish-like automatic isolation and translation of any speech around you * Privacy from being able to communicate online or record videos without revealing your real voice, which I think is why many (myself included) currently resort to text-only * New forms of interactive media - customised movies, audio dramas where the listener plays a role, videogame NPCs that react with more than just prerecorded lines, etc. * And of course: memes, satire, and parody I appreciate HN's general view on technologies like encrypted messaging - not falling into "we need to ban this now because pedophiles could use it" hysteria. But for anything involving machine learning, I'm concerned how often the hacker mentality seems to go out the window and we instead get people advocating for it to be made illegal to host the code, for instance.
• casey2 2y ago
  I like tools like these cause they make zero trust default even more obvious, and their "pretty limited use" is saving people hours of work.
• anonzzzies 2y ago
  They are pretty good for leaving messages for my blind friend. I generally find calling / voice texts a waste of time (I type and read far faster than I talk or listen, not to mention the ability to reread etc), but my blind friend prefers getting voice messages when on his phone and this works for us. I type and send and when he comes back with something, Whisper makes it into text for me.
• mistercow 2y ago
  It’s weird to me that people look at a technology and then assume from their reckoning that they know all the uses for that technology immediately. Most technological progress happens because someone notices a creative use for something that already exists which nobody else has noticed.