8 ms·

This article only covers the musical aspects of AI voice cloning, but there's another dynamic to AI voice cloning that's more complicated: replacing general voi

This article only covers the musical aspects of AI voice cloning, but there's another dynamic to AI voice cloning that's more complicated: replacing general voice actors in movies/video games/anime (example: https://www.axios.com/2023/07/24/ai-voice-actors-victoria-atkin-assassins-creed https://www.axios.com/2023/07/24/ai-voice-actors-victoria-at... ) Unlike musicians who can't be replaced without significant postprocessing, have enough money to not be impacted by competition, and have legal muscle, voice over artists: - Can be reproduced with good-enough results from out-of-the-box voice cloning settings on ElevenLabs or an open source equivalent (Bark, VALL-E X) - Are already underpaid for their work as-is - Have no legal ownership of their voice since they are contractors, and their voicework is owned by their clients who may not be as incentivised in protecting the VO. I want to write a blog post about it but I suspect most people on Hacker News won't be interested in a treatise on the cultural impacts of the voicework in Persona 5 and Genshin Impact.

aaroninsf 3y ago
<raises hand> I am
supriyo-biswas 3y ago
HN isn't the only community to write for. While most people here seem to be unsympathetic to such job concerns, unconventional articles do hit the front page from time to time. I'd like to read it, in any case.
- pixl97 3y ago
  The get rich at any cost type like to post on these articles at a higher rate I think. When you read a larger and broad range of HN posts you see a substantial part of the population here has concerns about this.
- rcarr 3y ago
  +1, I would also like to read it
  - galangalalgol 3y ago
    I would as well. It isn't that I'm unsympathetic, it is just that we haven't outlawed technology that put others out of work, and I'm curious why we would decide as a society this time should be different. If there are good reasons I want to know.
    - mschuster91 3y ago
      Putting people out of work is one thing, that's bad enough and societies should take care to guide change and support those affected. The danger behind AI and other manipulative technology is that it erodes trust. We already have serious issues with trust in media, and not just the obvious cases of Russian/Chinese propaganda, but also stuff like kids getting anorexia from extremely photoshopped advertising. Add AI on top and no one can be certain about anything anymore. Say someone distributes a fake "recording" of the US President calling for glassing Moscow, or the Serbian President declaring war on Kosovo? That has the potential to actually cost lives on a massive scale.
      - galangalalgol 3y ago
        Yeah, all that is bad, but those consequences are already here aren't they? Restricting further research just means it will be done in clandestine government labs like chemical or biological weapons except with equipment costs orders of magnitude lower. I can imagine policies that would save the jobs of voice actors, but none that would prevent the wave of deepfake propaganda that is coming.
foobarian 3y ago
It saddens me because of how much impact they had on my family as we played through the story line in Genshin and immersed in the world. At some point we met a few of the voice actors at a convention and they were like stars to us, while I'm sure their circumstances are as you describe.
zerojames 3y ago
I am interested! You should write about what you find interesting; never worry if it will interest a particular group.
sumtechguy 3y ago
What I find interesting is this aspect that eventually, these companies will hire some college kids who needs a couple thousand bucks and a free pizza. Have them read the right scripts. Sign the right 'give everything away' contract and just forever use their voice. Or do it sneaky. Have a voice assistant and in your ToS 'we can use a copy of your voice for anything'. The existing voice actors will be just out of work. There will be a small cadre of groups that want real voice. But for some projects that will not be that important. Its going to get crazy.
- minimaxir 3y ago
  Recent voice models by OpenAI, Meta, and ElevenLabs all state upfront they work with paid professional voice actors, so this space will get intetesting fast.
- hiccuphippo 3y ago
  Mozilla has a voice data project where people already do it for free(dom) ;) https://commonvoice.mozilla.org/en https://commonvoice.mozilla.org/en
- HappyDaoDude 3y ago
  I have said this will initially be sold as a feature on things like Audiobooks. Pick your book, pick your reader and away it goes. The Diary of Anne Frank read by Gilbert Gottfried.
  - trafficante 3y ago
    Not sure if your hypothetical was meant to be a reference to the absolutely hilarious classic “Gilbert Gottfried reads 50 Shades of Gray”, but it has me wondering how much of the inherent comedy comes from “the voice” and how much comes from the idea that the man himself sat down and recorded those lines. https://youtu.be/XkLqAlIETkA https://youtu.be/XkLqAlIETkA (Extremely NSFW without headphones)
    - totetsu 3y ago
ImprobableTruth 3y ago
Voices are uncopyrightable, but impersonation isn't legal (see Midler v. Ford, for a notable case), so I don't think the situation is totally clear.
- deepsun 3y ago
  > voice actors are fearing that the ability for generative AI to replicate their voices may cost them work I'm not sure how to feel about that. I'm against the idea that some people "deserve" being paid for being lucky born with an interesting voice. On the other hand, the world always worked like that. And, say, hard-working farmer or doctor were also lucky being born with necessary traits to make for their living, while others weren't.
  - minimaxir 3y ago
    Voice acting is more than just talking into a microphone. It's a skill not limited to the quality of voice.
    - deepsun 3y ago
      A lot of skills are not simple, but computers have taken over them anyways. For example, financial bookkeeping is not just writing and storing the books, it's a professional skill with many tricks to learn. However, databases and spreadsheets have taken the major part from those jobs. Same could be said about programmers who learned the skill of programming Assembly language. Or performing -- vinyl records and CDs has largely taken over orchestras and traveling musicians. I would vote for it only if it somehow encouraged voice actors to experiment and create new interesting styles. Kinda like patents were designed to do -- encourage inventors (although recently it became controversial in IT world).
      - deleted 3y ago
        [deleted]
    - vunderba 3y ago
      You could have made that argument more effectively in the past when voice actors had to be able to mimic multiple voices (Dan Castlenetta, Mel Blanc, etc.). Nowadays, we're seeing more and more shows where the voices of the characters are just... the normal voice of the voice actor. Of course it's not totally devoid of skill, you need to be able to emote, inflect, and convey emotion, but the bar is far far lower.
dylan604 3y ago
> and their voicework is owned by their clients who may not be as incentivised in protecting the VO. The work product produced by their voice for fulfilling the contract is owned. No corp owns someone else's voice.
- minimaxir 3y ago
  They don't own the voice, but they own the vocal performance, which ends up being a meaningless legal distinction in practice. It's one reason why VAs rarely take fan requests for a character they voice.
  - dylan604 3y ago
    If they are using their real voice, then they kind of screwed themselves. If they are performing a character voice, then at least they only lose out on that kind of work. I'm guessing contracts will need to be updated to say that a character's voice made from AI can't be used so a completely different production cannot say they have the actor attached for publicity purposes.
- Jeff_Brown 3y ago
  Porperty is a bundle of rights, and often hard to pin down. In the case of voices, if a company owns enough of your data to train a good simulacrum, and they have the right to do it, then they kind of do own your voice -- or more precisely, a damn good substitute.
  - minimaxir 3y ago
    Case in point, Luke Skywalker / Darth Vader in the D+ series: https://www.vanityfair.com/hollywood/2022/09/darth-vaders-voice-emanated-from-war-torn-ukraine https://www.vanityfair.com/hollywood/2022/09/darth-vaders-vo... > Belyaev is a 29-year-old synthetic-speech artist at the Ukrainian start-up Respeecher, which uses archival recordings and a proprietary A.I. algorithm to create new dialogue with the voices of performers from long ago. The company worked with Lucasfilm to generate the voice of a young Luke Skywalker for Disney+’s The Book of Boba Fett, and the recent Obi-Wan Kenobi series tasked them with making Darth Vader sound like James Earl Jones’s dark side villain from 45 years ago, now that Jones’s voice has altered with age and he has stepped back from the role.
raytopia 3y ago
I'd be interested. Most likey you'd see a lot of people saying that somehow getting rid of voice actors is good for "progress". Whatever that means. Random aside someone really needs to make a hackernews that focuses more on game development and other arts so blog posts like your talking about would have a proper community to discuss them with.
- Legend2440 3y ago
  Replacing voice actors with text-to-speech is good because it lets you do things voice actors can't: * Create dynamic new voice lines at runtime, for example game characters reacting to new situations. * Operate at a scale that's infeasible for humans, for example turning every ebook into an audiobook.
  - JohnFen 3y ago
    Which are, in my view, really minor advantages when compared to the disadvantages. Not only in terms of putting people out of work, but in terms of increasing the artifice of the world around us and decreasing its humanity.
    - Legend2440 3y ago
      "putting people out of work" by automating jobs is also a good thing. The amount of stuff humans can accomplish is strongly limited by the supply of workers. Automating one job frees them up to do other things.
      - JohnFen 3y ago
        > "putting people out of work" by automating jobs is also a good thing Unless you're one of the people out of work. And even if you don't care anything about them, if there's enough of them then the resulting unrest will be your problem anyway.
    - cm2012 3y ago
      There's little nothing more important to the happiness of humanity than increased productivity per capita. That sounds crazy but when you think about it it's true.
EGreg 3y ago
Please do. Some of us critique capitalism
rcarr 3y ago
It's sad if the only way voice actors are going to be able to make a living is by doing stuff like Critical Role on Youtube. I love Critical Role but it likely wouldn't be the same if those guys hadn't spent years honing their craft. Watching people play RPGs online has replaced a lot of my streaming viewing now, but the market is much smaller and I imagine it can only sustain a much smaller pool of creatives than the current voice over market can.
GuB-42 3y ago
Interesting note: many Vocaloids (most notably Hatsune Miku) are sampled from voice actors rather than singers. Singers didn't want software clones, but voices actors are fair game.
sublinear 3y ago
I have a different take on this. AI voice is cheaper, but it's also a more boring and generic performance. There is zero progress made towards any sort of creative AI that produces good unique work. The market for this then is small businesses who can't afford a professional voice actor. AI is opening up new markets, not killing the jobs of the truly talented.
- chefandy 3y ago
  This is the case for all generative "art." The people at the high end will still get paid well. The people who specialize in more utilitarian or low budget tasks in higher volume will take the biggest hit. Nobody who'd planned on hiring Morgan Freeman to do a voice over will be tempted to use AI Morgan Freeman instead.
- aussieguy1234 3y ago
  The MVP might have the free "good enough" AI voiceover, it takes less money to bootstrap a new product that way. The real product would have a real voice over actor paid for with VC money.
- matteoraso 3y ago
  >There is zero progress made towards any sort of creative AI that produces good unique work. It's only been a year. Give it some time and I'm sure AI will have much better results. Right now, you can get some of that unique work by finetuning the AI off of a person's existing portfolio.

aaroninsf 3y ago

<raises hand> I am

supriyo-biswas 3y ago

HN isn't the only community to write for. While most people here seem to be unsympathetic to such job concerns, unconventional articles do hit the front page from time to time. I'd like to read it, in any case.

pixl97 3y ago
The get rich at any cost type like to post on these articles at a higher rate I think. When you read a larger and broad range of HN posts you see a substantial part of the population here has concerns about this.
rcarr 3y ago
+1, I would also like to read it
- galangalalgol 3y ago
  I would as well. It isn't that I'm unsympathetic, it is just that we haven't outlawed technology that put others out of work, and I'm curious why we would decide as a society this time should be different. If there are good reasons I want to know.
  - mschuster91 3y ago
    Putting people out of work is one thing, that's bad enough and societies should take care to guide change and support those affected. The danger behind AI and other manipulative technology is that it erodes trust. We already have serious issues with trust in media, and not just the obvious cases of Russian/Chinese propaganda, but also stuff like kids getting anorexia from extremely photoshopped advertising. Add AI on top and no one can be certain about anything anymore. Say someone distributes a fake "recording" of the US President calling for glassing Moscow, or the Serbian President declaring war on Kosovo? That has the potential to actually cost lives on a massive scale.
    - galangalalgol 3y ago
      Yeah, all that is bad, but those consequences are already here aren't they? Restricting further research just means it will be done in clandestine government labs like chemical or biological weapons except with equipment costs orders of magnitude lower. I can imagine policies that would save the jobs of voice actors, but none that would prevent the wave of deepfake propaganda that is coming.

foobarian 3y ago

It saddens me because of how much impact they had on my family as we played through the story line in Genshin and immersed in the world. At some point we met a few of the voice actors at a convention and they were like stars to us, while I'm sure their circumstances are as you describe.

zerojames 3y ago

I am interested! You should write about what you find interesting; never worry if it will interest a particular group.

sumtechguy 3y ago

What I find interesting is this aspect that eventually, these companies will hire some college kids who needs a couple thousand bucks and a free pizza. Have them read the right scripts. Sign the right 'give everything away' contract and just forever use their voice. Or do it sneaky. Have a voice assistant and in your ToS 'we can use a copy of your voice for anything'. The existing voice actors will be just out of work. There will be a small cadre of groups that want real voice. But for some projects that will not be that important. Its going to get crazy.

minimaxir 3y ago
Recent voice models by OpenAI, Meta, and ElevenLabs all state upfront they work with paid professional voice actors, so this space will get intetesting fast.
hiccuphippo 3y ago
Mozilla has a voice data project where people already do it for free(dom) ;) https://commonvoice.mozilla.org/en https://commonvoice.mozilla.org/en
HappyDaoDude 3y ago
I have said this will initially be sold as a feature on things like Audiobooks. Pick your book, pick your reader and away it goes. The Diary of Anne Frank read by Gilbert Gottfried.
- trafficante 3y ago
  Not sure if your hypothetical was meant to be a reference to the absolutely hilarious classic “Gilbert Gottfried reads 50 Shades of Gray”, but it has me wondering how much of the inherent comedy comes from “the voice” and how much comes from the idea that the man himself sat down and recorded those lines. https://youtu.be/XkLqAlIETkA https://youtu.be/XkLqAlIETkA (Extremely NSFW without headphones)
  - totetsu 3y ago

ImprobableTruth 3y ago

Voices are uncopyrightable, but impersonation isn't legal (see Midler v. Ford, for a notable case), so I don't think the situation is totally clear.

deepsun 3y ago
> voice actors are fearing that the ability for generative AI to replicate their voices may cost them work I'm not sure how to feel about that. I'm against the idea that some people "deserve" being paid for being lucky born with an interesting voice. On the other hand, the world always worked like that. And, say, hard-working farmer or doctor were also lucky being born with necessary traits to make for their living, while others weren't.
- minimaxir 3y ago
  Voice acting is more than just talking into a microphone. It's a skill not limited to the quality of voice.
  - deepsun 3y ago
    A lot of skills are not simple, but computers have taken over them anyways. For example, financial bookkeeping is not just writing and storing the books, it's a professional skill with many tricks to learn. However, databases and spreadsheets have taken the major part from those jobs. Same could be said about programmers who learned the skill of programming Assembly language. Or performing -- vinyl records and CDs has largely taken over orchestras and traveling musicians. I would vote for it only if it somehow encouraged voice actors to experiment and create new interesting styles. Kinda like patents were designed to do -- encourage inventors (although recently it became controversial in IT world).
    - deleted 3y ago
      [deleted]
  - vunderba 3y ago
    You could have made that argument more effectively in the past when voice actors had to be able to mimic multiple voices (Dan Castlenetta, Mel Blanc, etc.). Nowadays, we're seeing more and more shows where the voices of the characters are just... the normal voice of the voice actor. Of course it's not totally devoid of skill, you need to be able to emote, inflect, and convey emotion, but the bar is far far lower.

dylan604 3y ago

> and their voicework is owned by their clients who may not be as incentivised in protecting the VO. The work product produced by their voice for fulfilling the contract is owned. No corp owns someone else's voice.

minimaxir 3y ago
They don't own the voice, but they own the vocal performance, which ends up being a meaningless legal distinction in practice. It's one reason why VAs rarely take fan requests for a character they voice.
- dylan604 3y ago
  If they are using their real voice, then they kind of screwed themselves. If they are performing a character voice, then at least they only lose out on that kind of work. I'm guessing contracts will need to be updated to say that a character's voice made from AI can't be used so a completely different production cannot say they have the actor attached for publicity purposes.
Jeff_Brown 3y ago
Porperty is a bundle of rights, and often hard to pin down. In the case of voices, if a company owns enough of your data to train a good simulacrum, and they have the right to do it, then they kind of do own your voice -- or more precisely, a damn good substitute.
- minimaxir 3y ago
  Case in point, Luke Skywalker / Darth Vader in the D+ series: https://www.vanityfair.com/hollywood/2022/09/darth-vaders-voice-emanated-from-war-torn-ukraine https://www.vanityfair.com/hollywood/2022/09/darth-vaders-vo... > Belyaev is a 29-year-old synthetic-speech artist at the Ukrainian start-up Respeecher, which uses archival recordings and a proprietary A.I. algorithm to create new dialogue with the voices of performers from long ago. The company worked with Lucasfilm to generate the voice of a young Luke Skywalker for Disney+’s The Book of Boba Fett, and the recent Obi-Wan Kenobi series tasked them with making Darth Vader sound like James Earl Jones’s dark side villain from 45 years ago, now that Jones’s voice has altered with age and he has stepped back from the role.

raytopia 3y ago

I'd be interested. Most likey you'd see a lot of people saying that somehow getting rid of voice actors is good for "progress". Whatever that means. Random aside someone really needs to make a hackernews that focuses more on game development and other arts so blog posts like your talking about would have a proper community to discuss them with.

Legend2440 3y ago
Replacing voice actors with text-to-speech is good because it lets you do things voice actors can't: * Create dynamic new voice lines at runtime, for example game characters reacting to new situations. * Operate at a scale that's infeasible for humans, for example turning every ebook into an audiobook.
- JohnFen 3y ago
  Which are, in my view, really minor advantages when compared to the disadvantages. Not only in terms of putting people out of work, but in terms of increasing the artifice of the world around us and decreasing its humanity.
  - Legend2440 3y ago
    "putting people out of work" by automating jobs is also a good thing. The amount of stuff humans can accomplish is strongly limited by the supply of workers. Automating one job frees them up to do other things.
    - JohnFen 3y ago
      > "putting people out of work" by automating jobs is also a good thing Unless you're one of the people out of work. And even if you don't care anything about them, if there's enough of them then the resulting unrest will be your problem anyway.
  - cm2012 3y ago
    There's little nothing more important to the happiness of humanity than increased productivity per capita. That sounds crazy but when you think about it it's true.

EGreg 3y ago

Please do. Some of us critique capitalism

rcarr 3y ago

It's sad if the only way voice actors are going to be able to make a living is by doing stuff like Critical Role on Youtube. I love Critical Role but it likely wouldn't be the same if those guys hadn't spent years honing their craft. Watching people play RPGs online has replaced a lot of my streaming viewing now, but the market is much smaller and I imagine it can only sustain a much smaller pool of creatives than the current voice over market can.

GuB-42 3y ago

Interesting note: many Vocaloids (most notably Hatsune Miku) are sampled from voice actors rather than singers. Singers didn't want software clones, but voices actors are fair game.

sublinear 3y ago

I have a different take on this. AI voice is cheaper, but it's also a more boring and generic performance. There is zero progress made towards any sort of creative AI that produces good unique work. The market for this then is small businesses who can't afford a professional voice actor. AI is opening up new markets, not killing the jobs of the truly talented.

chefandy 3y ago
This is the case for all generative "art." The people at the high end will still get paid well. The people who specialize in more utilitarian or low budget tasks in higher volume will take the biggest hit. Nobody who'd planned on hiring Morgan Freeman to do a voice over will be tempted to use AI Morgan Freeman instead.
aussieguy1234 3y ago
The MVP might have the free "good enough" AI voiceover, it takes less money to bootstrap a new product that way. The real product would have a real voice over actor paid for with VC money.
matteoraso 3y ago
>There is zero progress made towards any sort of creative AI that produces good unique work. It's only been a year. Give it some time and I'm sure AI will have much better results. Right now, you can get some of that unique work by finetuning the AI off of a person's existing portfolio.