5 ms·

I think it's plausible that it can roughly clone the timbre that my exact vocal and sinus geometry produces, then apply that to its internal representation of a

I think it's plausible that it can roughly clone the timbre that my exact vocal and sinus geometry produces, then apply that to its internal representation of a "baseline human voice". However, I don't believe that it can capture exactly every nuance in articulation, emotional expression, intonation. I simply can't imagine even a voice actor iterating through "the full range" in merely 15 seconds. And what about mannerisms, like pronouncing "you all" as "y'all"? Ending every sentence by affixing the "uh" sound to the last word-uh? Or how they sound a sigh or laugh? In Dutch for instance, there's 3 ways to pronounce the R, and everyone has a certain combination of when they use which kind.

jstummbillig 3y ago
Sure. Edge cases will be infinite.
xbmcuser 3y ago
A lot of people miss the forest for the trees. They can clone a voice using 15 seconds of audio. So how good the clone will be when they have 10 min, 30 min or larger sample size. And it will probably keep getting better
marcusverus 3y ago
> And what about mannerisms, like pronouncing "you all" as "y'all"? In the US this is highly determined by location, to the point where the New York Times even built a cool quiz [0] that can guess where you're from pretty accurately based on a few dozen vocab questions. So there's a dataset out there that would allow you to handle most of this variance by simply plugging in a ZIP code for the person you're spoofing! [0]https://archive.is/yodpA https://archive.is/yodpA
criddell 3y ago
Does it really matter if they have not produced a perfect clone but merely a convincing clone?
numpad0 3y ago
It's often not necessary to sample all keys on a piano to make a passable piano sound generator, though that will have its own limitations. A lot of these generators rely heavily on the base model and are worse at generating e.g. Dutch accented English and/or Dutch accented Dutch. But it works great for e.g. Californian accented English with unique voice.
verticalscaler 3y ago
I'll speculate "what is really happening" - Think of a voice like a musical instrument. This learned to be a Shazam for human voices instead of songs. And just like you can figure out by ear a snippet of music was made on a Yamaha DX7, seems 15 seconds is enough to narrow it down to a reasonably small set of vocoders that can recreate the sample given.

jstummbillig 3y ago

Sure. Edge cases will be infinite.

xbmcuser 3y ago

A lot of people miss the forest for the trees. They can clone a voice using 15 seconds of audio. So how good the clone will be when they have 10 min, 30 min or larger sample size. And it will probably keep getting better

marcusverus 3y ago

> And what about mannerisms, like pronouncing "you all" as "y'all"? In the US this is highly determined by location, to the point where the New York Times even built a cool quiz [0] that can guess where you're from pretty accurately based on a few dozen vocab questions. So there's a dataset out there that would allow you to handle most of this variance by simply plugging in a ZIP code for the person you're spoofing! [0]https://archive.is/yodpA https://archive.is/yodpA

criddell 3y ago

Does it really matter if they have not produced a perfect clone but merely a convincing clone?

numpad0 3y ago

It's often not necessary to sample all keys on a piano to make a passable piano sound generator, though that will have its own limitations. A lot of these generators rely heavily on the base model and are worse at generating e.g. Dutch accented English and/or Dutch accented Dutch. But it works great for e.g. Californian accented English with unique voice.

verticalscaler 3y ago

I'll speculate "what is really happening" - Think of a voice like a musical instrument. This learned to be a Shazam for human voices instead of songs. And just like you can figure out by ear a snippet of music was made on a Yamaha DX7, seems 15 seconds is enough to narrow it down to a reasonably small set of vocoders that can recreate the sample given.