7 ms·

I don't usually expect much from demos like this, but I'm kind of surprised how impressive the results currently are. They're definitely not perfect, you're def

I don't usually expect much from demos like this, but I'm kind of surprised how impressive the results currently are. They're definitely not perfect, you're definitely getting some odd clipping and noise, but this shows a large amount of promise. Being able to generate voices for games would enable a lot of interesting indie projects. IMO people should be paying more attention the market implications of products like this than to the social implications. There are a lot of projects that just aren't really feasible right now that could be if this kind of technology was more polished and generally available for commercial/self-hosted use. And in those cases, you don't even need to do inference, makers will likely be willing to mark up their scripts themselves. Anyway I digress. Congrats, this is really cool!

Pfhreak 6y ago
> people should be paying more attention the market implications of products like this than to the social implications. People will absolutely suffer harm from this tech, but hey, think about the dollars that could be made! No, we should absolutely be paying more attention to the social implications.
- danShumway 6y ago
  Eh, this technology currently falls very squarely into the category of "almost good enough that I could use it for a creative project, but not nearly good enough that you're going to be able to convince me that the results aren't generated." I'm not primarily interested about the dollars, I'm interested in allowing communities to do creative things. I think people are looking at this tech like it's only going to be used for deepfakes, and they're underestimating the extent it's going to be used to create voice-acted game mods, animations, anonymization tools, and other creative/helpful projects. If you're really worried about this stuff though, you can take some comfort in the fact that by far the worst examples on the site are of real-world voices. This is currently technology that as far as I can see is far more suited for generating new voices or voicing cartoon characters with well-defined patterns/inflections than it is for imitating the president.
  - significant5 6y ago
    I might be misunderstanding you, but there are no real-world voices on the site? All of them are of characters.
    - danShumway 6y ago
      I see a pretty linear drop in quality from Glados to Spongebob to Twilight Sparkle to the narrator from Stanley Parable to the 10th Doctor. It seems to struggle more and more as the voices get less cartoony/exaggerated.
      - significant5 6y ago
        I'm not too sure about that. From my testing, Fluttershy, Applejack, Twilight, Chrysalis, Rise, and Kyu (and a bunch of other characters that I'm surely forgetting) seem to perform phenomenally well. Especially Chrysalis, her emotions are extremely believable, and Fluttershy/Applejack/Rise/Kyu have almost zero noise for every generation. This might be the most impressive site I've ever seen. Oh, I somehow forgot all of the TF2 characters. Some of them do struggle (Medic the most, I think) but everyone else seems incredibly good. And the Daria characters, too. Honestly, the vast majority of characters are already near-perfect.
        danShumway 6y ago
        Hrm. Well, I can't really argue with that beyond that my standards on perfect might be different. I think some of the best voices they have are characters like Twilight, she shows a ton of promise. But as it stands right now, I would still at least hesitate to use Twilight's voice in a project unless I didn't have other options. Chrysalis's voice is good, but again, is an exaggerated cartoon character with a large amount of inflection. I would not use her voice in her current state without a lot of post-processing. Someone like the Spy I would consider to be unusable, it sounds to me like the character needs to clear their throat or something, it's got a lot of strange artifacts. I definitely would consider the 10th Doctor unusable, even for just a hobby project or a voice assistant. But... I don't know, maybe this is subjective. I can't just tell you that what you're hearing is wrong, if you like the results then you like the results :) And again, I don't want to detract from how impressive they are. They are incredibly impressive, particularly because of how characters like Chrysalis emote. Extremely promising. But I still think there's a difference between 'impressive' and 'believable deepfake'.
- C19is20 6y ago
  Musicians Union?
Ajedi32 6y ago
I wonder if there are any legal concerns with using the voices of well known characters/actors like this in a commercial context.
- danShumway 6y ago
  I don't think a voice can be copyrighted, but IANAL so you shouldn't bank on that. If a voice could be copyrighted, or if this was a trademark issue or something, I strongly suspect that this site would not fall under fair use regardless of whether or not it was commercial. But again, IANAL, so I don't feel confident making any kind of strong claim about that either.
  - dragonwriter 6y ago
    > I don't think a voice can be copyrighted, but IANAL so you shouldn't bank on that. The audio content (which includes voices) of the source work is copyrighted, and a mechanical transform of that work (which deep learning to mimic the voices clearly is) would seem to be a derivative in at least the literal sense.
    - thrill 6y ago
      IANAL and I would say no. Anyone is free to imitate any else. A machine doesn't make that different. It would be a violation to claim you were someone else while doing the imitation.

Pfhreak 6y ago

> people should be paying more attention the market implications of products like this than to the social implications. People will absolutely suffer harm from this tech, but hey, think about the dollars that could be made! No, we should absolutely be paying more attention to the social implications.

danShumway 6y ago
Eh, this technology currently falls very squarely into the category of "almost good enough that I could use it for a creative project, but not nearly good enough that you're going to be able to convince me that the results aren't generated." I'm not primarily interested about the dollars, I'm interested in allowing communities to do creative things. I think people are looking at this tech like it's only going to be used for deepfakes, and they're underestimating the extent it's going to be used to create voice-acted game mods, animations, anonymization tools, and other creative/helpful projects. If you're really worried about this stuff though, you can take some comfort in the fact that by far the worst examples on the site are of real-world voices. This is currently technology that as far as I can see is far more suited for generating new voices or voicing cartoon characters with well-defined patterns/inflections than it is for imitating the president.
- significant5 6y ago
  I might be misunderstanding you, but there are no real-world voices on the site? All of them are of characters.
  - danShumway 6y ago
    I see a pretty linear drop in quality from Glados to Spongebob to Twilight Sparkle to the narrator from Stanley Parable to the 10th Doctor. It seems to struggle more and more as the voices get less cartoony/exaggerated.
    - significant5 6y ago
      I'm not too sure about that. From my testing, Fluttershy, Applejack, Twilight, Chrysalis, Rise, and Kyu (and a bunch of other characters that I'm surely forgetting) seem to perform phenomenally well. Especially Chrysalis, her emotions are extremely believable, and Fluttershy/Applejack/Rise/Kyu have almost zero noise for every generation. This might be the most impressive site I've ever seen. Oh, I somehow forgot all of the TF2 characters. Some of them do struggle (Medic the most, I think) but everyone else seems incredibly good. And the Daria characters, too. Honestly, the vast majority of characters are already near-perfect.
      - danShumway 6y ago
        Hrm. Well, I can't really argue with that beyond that my standards on perfect might be different. I think some of the best voices they have are characters like Twilight, she shows a ton of promise. But as it stands right now, I would still at least hesitate to use Twilight's voice in a project unless I didn't have other options. Chrysalis's voice is good, but again, is an exaggerated cartoon character with a large amount of inflection. I would not use her voice in her current state without a lot of post-processing. Someone like the Spy I would consider to be unusable, it sounds to me like the character needs to clear their throat or something, it's got a lot of strange artifacts. I definitely would consider the 10th Doctor unusable, even for just a hobby project or a voice assistant. But... I don't know, maybe this is subjective. I can't just tell you that what you're hearing is wrong, if you like the results then you like the results :) And again, I don't want to detract from how impressive they are. They are incredibly impressive, particularly because of how characters like Chrysalis emote. Extremely promising. But I still think there's a difference between 'impressive' and 'believable deepfake'.
C19is20 6y ago
Musicians Union?

Ajedi32 6y ago

I wonder if there are any legal concerns with using the voices of well known characters/actors like this in a commercial context.

danShumway 6y ago
I don't think a voice can be copyrighted, but IANAL so you shouldn't bank on that. If a voice could be copyrighted, or if this was a trademark issue or something, I strongly suspect that this site would not fall under fair use regardless of whether or not it was commercial. But again, IANAL, so I don't feel confident making any kind of strong claim about that either.
- dragonwriter 6y ago
  > I don't think a voice can be copyrighted, but IANAL so you shouldn't bank on that. The audio content (which includes voices) of the source work is copyrighted, and a mechanical transform of that work (which deep learning to mimic the voices clearly is) would seem to be a derivative in at least the literal sense.
  - thrill 6y ago
    IANAL and I would say no. Anyone is free to imitate any else. A machine doesn't make that different. It would be a violation to claim you were someone else while doing the imitation.