3 ms·

I'm with you on this. I can't honestly think a good use case for the average user to generate audio this way. Maybe some niche use case in like movie or tv prod

I'm with you on this. I can't honestly think a good use case for the average user to generate audio this way. Maybe some niche use case in like movie or tv production where you can generate a missing line without flying in an actor or something. Or maybe for generating dialogues for videogames. But those are business use cases, not things for the genral public.

RobotToaster 4y ago
Pretty much any hobby video game development or animation?
- arroz 4y ago
  So basically stealing other people voice for you hobby? Great!
  - skeaker 4y ago
    I'm 100% certain that people will offer up their own voices as open-source. I would be happy to, albeit maybe anonymously.
  - exolymph 4y ago
    Who says it has to be someone else's voice?
    - manuelmoreale 4y ago
      I mean, if it is your voice, can’t you just record it directly? Isn’t that better than going through an artificial middle man that has to be primed first with some recording of yours?
      - pseudo0 4y ago
        It would be a huge time saver. Typically a pro does around 100 lines per hour, an amateur doing multiple takes would be significantly slower. So a character with 1000 lines could easily be 20-30 hours of work, just for a first draft. It would be pretty amazing to be able to just revise the script and auto-generate a new recording, even if the quality is only 90% there. Just like with image generation models, this will massively raise the bar for what amateurs can do with a limited budget and limited time. It's hard to justify spending thousands of dollars on voice acting and art for a hobby project, but now amateurs can get something that is 90% there and substitute professional work if the project takes off.
imjonse 4y ago
Artificially generated but faithful to the original voice for people losing theirs or being unable to speak for various medical reasons. Obviously the fun/deceiving use-cases are much more numerous.
biomcgary 4y ago
When my sons were young, I would tell them elaborate stories where they were the main characters. I recorded most of the stories, but it is full of verbal fillers (um,ahh), since I was making it up as I went. I would love to convert the audio to text with Whisper, filter out fillers and then output the cleaned up version in my own voice. I could see this type of workflow being very popular with podcasters.
- sebastiennight 4y ago
  You can already do what you are aiming for, without the transcription part (removing fillers with word-based filters does not give great results compared to removing with voice-based filters). You can use Descript, CleanVoice, and other tools to achieve exactly what you just said, in a few minutes, from just the original recording.
- peteforde 4y ago
  You should absolutely do this, but please skip the Whisper. The reason is that if you speak with lots of verbal fillers, that's actually an important part of how you sound to other people. It makes sense to clean up audio for a podcast, but not for your great grandchildren. A voice cloner doesn't care that you say "um" too much. It's parsing audio for phonemes.
rtp4me 4y ago
Personally, I would find this very useful. I (used to) create internal tech training videos for our organization and would routinely stumble when doing voice overlay. Even though everything was scripted out, it took lots of editing time to get the audio and video aligned without the vocal stumbling (ahs, ehs, silence, voice redos, etc). Just my $0.02
notafraudster 4y ago
My Dad died before my kids were born. When he got cancer, he recorded himself reading "The Night Before Christmas", which is about enough audio for the high quality version of this technology. Is it ghoulish on my part to want to hear his voice again or for my kids to hear it? Maybe. Do I really care what you think (or really what _I_ think) about that? No.
- ss108 4y ago
  Sorry to hear. Hope your kids enjoy his recording. But yes, it would be weird to generate more stuff spoken by your father by using this technology. And beyond that, what's even the point? It's not your dad.
- kossTKR 4y ago
  It doesn't really sound healthy to generate content of loved ones. Yes in a few years you would be able to generate a complete avatar of someone, but it isn't them, and i think it will mess with you mentally.
- peteforde 4y ago
  People who would tell you not to use your recorded audio to create more simulations of your father speaking are the same sort of folks with strong opinions about what other people do in the bedroom. I happen to be someone who believes that it's wonderful your dad left you with this artifact. It was a touching sentiment then, and now it can serve his obvious purpose many times over. He didn't record himself as a side-effect of disease, or because he loved that particular story in the sound of his voice. He wanted people in the future to be able to hear what he sounded like! Given that he could not have foreseen voice cloning (and therefore not explicitly asked for it) I cannot think of a more obvious example of someone wanting their voice to survive them. I wish more folks would record The Night Before Christmas.
- manuelmoreale 4y ago
  You shouldn’t care what I think. You shouldn’t care what anyone here thinks. Creating fake memories is not something I’d ever consider doing but that’s just me.

RobotToaster 4y ago

Pretty much any hobby video game development or animation?

arroz 4y ago
So basically stealing other people voice for you hobby? Great!
- skeaker 4y ago
  I'm 100% certain that people will offer up their own voices as open-source. I would be happy to, albeit maybe anonymously.
- exolymph 4y ago
  Who says it has to be someone else's voice?
  - manuelmoreale 4y ago
    I mean, if it is your voice, can’t you just record it directly? Isn’t that better than going through an artificial middle man that has to be primed first with some recording of yours?
    - pseudo0 4y ago
      It would be a huge time saver. Typically a pro does around 100 lines per hour, an amateur doing multiple takes would be significantly slower. So a character with 1000 lines could easily be 20-30 hours of work, just for a first draft. It would be pretty amazing to be able to just revise the script and auto-generate a new recording, even if the quality is only 90% there. Just like with image generation models, this will massively raise the bar for what amateurs can do with a limited budget and limited time. It's hard to justify spending thousands of dollars on voice acting and art for a hobby project, but now amateurs can get something that is 90% there and substitute professional work if the project takes off.

imjonse 4y ago

Artificially generated but faithful to the original voice for people losing theirs or being unable to speak for various medical reasons. Obviously the fun/deceiving use-cases are much more numerous.

biomcgary 4y ago

When my sons were young, I would tell them elaborate stories where they were the main characters. I recorded most of the stories, but it is full of verbal fillers (um,ahh), since I was making it up as I went. I would love to convert the audio to text with Whisper, filter out fillers and then output the cleaned up version in my own voice. I could see this type of workflow being very popular with podcasters.

sebastiennight 4y ago
You can already do what you are aiming for, without the transcription part (removing fillers with word-based filters does not give great results compared to removing with voice-based filters). You can use Descript, CleanVoice, and other tools to achieve exactly what you just said, in a few minutes, from just the original recording.
peteforde 4y ago
You should absolutely do this, but please skip the Whisper. The reason is that if you speak with lots of verbal fillers, that's actually an important part of how you sound to other people. It makes sense to clean up audio for a podcast, but not for your great grandchildren. A voice cloner doesn't care that you say "um" too much. It's parsing audio for phonemes.

rtp4me 4y ago

Personally, I would find this very useful. I (used to) create internal tech training videos for our organization and would routinely stumble when doing voice overlay. Even though everything was scripted out, it took lots of editing time to get the audio and video aligned without the vocal stumbling (ahs, ehs, silence, voice redos, etc). Just my $0.02

notafraudster 4y ago

My Dad died before my kids were born. When he got cancer, he recorded himself reading "The Night Before Christmas", which is about enough audio for the high quality version of this technology. Is it ghoulish on my part to want to hear his voice again or for my kids to hear it? Maybe. Do I really care what you think (or really what _I_ think) about that? No.

ss108 4y ago
Sorry to hear. Hope your kids enjoy his recording. But yes, it would be weird to generate more stuff spoken by your father by using this technology. And beyond that, what's even the point? It's not your dad.
kossTKR 4y ago
It doesn't really sound healthy to generate content of loved ones. Yes in a few years you would be able to generate a complete avatar of someone, but it isn't them, and i think it will mess with you mentally.
peteforde 4y ago
People who would tell you not to use your recorded audio to create more simulations of your father speaking are the same sort of folks with strong opinions about what other people do in the bedroom. I happen to be someone who believes that it's wonderful your dad left you with this artifact. It was a touching sentiment then, and now it can serve his obvious purpose many times over. He didn't record himself as a side-effect of disease, or because he loved that particular story in the sound of his voice. He wanted people in the future to be able to hear what he sounded like! Given that he could not have foreseen voice cloning (and therefore not explicitly asked for it) I cannot think of a more obvious example of someone wanting their voice to survive them. I wish more folks would record The Night Before Christmas.
manuelmoreale 4y ago
You shouldn’t care what I think. You shouldn’t care what anyone here thinks. Creating fake memories is not something I’d ever consider doing but that’s just me.