3 ms·
It sounds to me like the edits are accomplished by taking the original text and feeding back through the model, so that the edited tracks contain zero original
by jerf 3y ago
It sounds to me like the edits are accomplished by taking the original text and feeding back through the model, so that the edited tracks contain zero original audio. Of course when your text precisely matches the training material it comes out pretty well, but not perfectly.
In late March 2024, this results in some noticeable distortion. Don't ask me whether you'll be able to notice by, say, September 2024 though.
It probably comes out less jarring than trying to insert the edits overall, but you still really want a clean voice sample going in. For instance, the Pulp Fiction voice had a background music track, and it does interesting things to the resulting voice once it has to start synthesizing novel text. (Second sample in the first table, Irving Ramses.) I wish they'd tacked on a bit more text to that one, I can just barely hear it's doing "weird things" but there's not a lot of time to analyze it before it's over.