4 ms·

I have a machine with two GPUs and a frozen OS with a tensorflow python app that clones voices, and I'd say the quality passes if you run it through a phone ban

I have a machine with two GPUs and a frozen OS with a tensorflow python app that clones voices, and I'd say the quality passes if you run it through a phone bandpass filter. I've had an idea to use propellerhead recycle to chop the output cloned voice into syllables, and then "play" the chopped parts in rhythm, through autotune. The issue is you get Eifel 65 sounding autotune if your base vocals are monotonic or way off key. The only way I can think of fixing this is to use something like audacity's pitch changer that doesn't affect the speed of the sample - rough the lyrical tones in with audacity/recycle, then autotune it where it needs to go. I'd like to say I'm too busy to get this workflow going, but mostly I'm lazy and someone else will do it first - and better - I can't improve the AI cloning software.

dkdbejwi383 5y ago
> The only way I can think of fixing this is to use something like audacity's pitch changer that doesn't affect the speed of the sample Check out Ableton Live’s “Simpler”
ethbr0 5y ago
You should message @echelon on HN. ;) https://vo.codes/ https://vo.codes/
- echelon 5y ago
  Oh hi! I've had a new version of this in the works for a few months. It'll launch maybe this weekend? It supports user-trained data sets, which should be pretty neat. Your approach sounds kind of like unit selection or vocaloid. Do it!! :)
  - Natsu 5y ago
    I'm honestly surprised that virtual celebs haven't taken over more. They could replace people in a lot of roles like newscasters and such.
    - minikomi 5y ago
      I'm seeing VTubers more and more on japanese (late night) TV green-screened in alongside regular people. It's coming soon.
      - echelon 5y ago
        I'm so stoked for this, and I totally agree. This is the future of entertainment. I'm working on mocap, photogrammetry, and compositing on Twitch, and I'm leveraging vocodes for subscriber interaction in the stream. Are either of you working in this domain? Want to chat?
        Natsu 5y ago
        I'd love to hear more about how this is done, but I've never done a thing with any of this other than trying to predict the future and seeing this as something that will take over eventually. I'm Natsu#7152 on Discord.
  - genewitch 5y ago
    Your gilbert gottfried is better than mine - in that it's more obviously gilbert, however, when he's doing interviews and not "in character" the one i have sounds closer to his actual voice.[0][1] I use(d) "Real time voice cloning" available on github. I have used your site before, i think that's actually the reason i wanted to find software to do it myself originally! I ended up using mine to do the announcements on all of the ham repeaters in Central Louisiana. I'm not sure how many are still active, but i had a couple of podcasters and two presidents doing the announcements for a while.[2] Nice work, wrapping it all in a website/service. I'll bookmark it! [0] https://soundcloud.com/djoutcold/gottfried1-1 https://soundcloud.com/djoutcold/gottfried1-1 [1] https://soundcloud.com/djoutcold/gilbertopiatecookies https://soundcloud.com/djoutcold/gilbertopiatecookies [2] https://soundcloud.com/djoutcold/trump-id-01 https://soundcloud.com/djoutcold/trump-id-01
    - echelon 5y ago
      Good results! Where did you get the voice samples? I love your use case for ham radio. That's brilliant!
      - genewitch 5y ago
        I just cut up youtube videos' audio to only have the voice i wanted to use. And thanks. I'm glad there's other people messing around with voice cloning, still.
stainforth 5y ago
How much time of audio or data do you need on average to train up something? I guess I'm wondering if I only had a few minutes of someone speaking, would that be enough?
- genewitch 5y ago
  The implementation i am using works best with several <30 second clips. I've tested it by cutting up 20 minute interviews to only have the person i care about, and it seems to be about the same as a half dozen 15 second clips.

dkdbejwi383 5y ago

> The only way I can think of fixing this is to use something like audacity's pitch changer that doesn't affect the speed of the sample Check out Ableton Live’s “Simpler”

ethbr0 5y ago

You should message @echelon on HN. ;) https://vo.codes/ https://vo.codes/

echelon 5y ago
Oh hi! I've had a new version of this in the works for a few months. It'll launch maybe this weekend? It supports user-trained data sets, which should be pretty neat. Your approach sounds kind of like unit selection or vocaloid. Do it!! :)
- Natsu 5y ago
  I'm honestly surprised that virtual celebs haven't taken over more. They could replace people in a lot of roles like newscasters and such.
  - minikomi 5y ago
    I'm seeing VTubers more and more on japanese (late night) TV green-screened in alongside regular people. It's coming soon.
    - echelon 5y ago
      I'm so stoked for this, and I totally agree. This is the future of entertainment. I'm working on mocap, photogrammetry, and compositing on Twitch, and I'm leveraging vocodes for subscriber interaction in the stream. Are either of you working in this domain? Want to chat?
      - Natsu 5y ago
        I'd love to hear more about how this is done, but I've never done a thing with any of this other than trying to predict the future and seeing this as something that will take over eventually. I'm Natsu#7152 on Discord.
- genewitch 5y ago
  Your gilbert gottfried is better than mine - in that it's more obviously gilbert, however, when he's doing interviews and not "in character" the one i have sounds closer to his actual voice.[0][1] I use(d) "Real time voice cloning" available on github. I have used your site before, i think that's actually the reason i wanted to find software to do it myself originally! I ended up using mine to do the announcements on all of the ham repeaters in Central Louisiana. I'm not sure how many are still active, but i had a couple of podcasters and two presidents doing the announcements for a while.[2] Nice work, wrapping it all in a website/service. I'll bookmark it! [0] https://soundcloud.com/djoutcold/gottfried1-1 https://soundcloud.com/djoutcold/gottfried1-1 [1] https://soundcloud.com/djoutcold/gilbertopiatecookies https://soundcloud.com/djoutcold/gilbertopiatecookies [2] https://soundcloud.com/djoutcold/trump-id-01 https://soundcloud.com/djoutcold/trump-id-01
  - echelon 5y ago
    Good results! Where did you get the voice samples? I love your use case for ham radio. That's brilliant!
    - genewitch 5y ago
      I just cut up youtube videos' audio to only have the voice i wanted to use. And thanks. I'm glad there's other people messing around with voice cloning, still.

stainforth 5y ago

How much time of audio or data do you need on average to train up something? I guess I'm wondering if I only had a few minutes of someone speaking, would that be enough?

genewitch 5y ago
The implementation i am using works best with several <30 second clips. I've tested it by cutting up 20 minute interviews to only have the person i care about, and it seems to be about the same as a half dozen 15 second clips.