4 ms·
They in the right direction, but wrong specifics: you can't maintain consistent prosody without context. The simplest examples are punctuation marks which chan
by BoorishBears 1y ago
They in the right direction, but wrong specifics: you can't maintain consistent prosody without context.
The simplest examples are punctuation marks which change your speech before you reach the mark, but the problem extends past sentence boundaries
For example:
"He didn't steal the green car. He borrowed it."
vs
"He didn't steal the green car. He stole the red one."
A natural speaker would slightly emphasize steal and borrowed in the 1st example, but emphasize green and red in the 2nd.
Or like when you're building a set:
"Peter called Mary."
vs
"John called Mary. Peter called Mary. Who didn't call Mary?"
-
These all sound like small nits but for naively stitched together TTS, at best they nudge the narration towards the uncanny valley (which may be acceptable for some usecases)... but at worst they make the model sound broken.
- zahlman 1y ago> The simplest examples are punctuation marks which change your speech before you reach the mark, but the problem extends past sentence boundaries I agree, but it seems unusual for this to matter past paragraph boundaries, and it sounds like there should be enough room for a full paragraph of context.
- BoorishBears 1y agoIt depends on the level of quality you're going for, but prosody even changes based on how related the preceeding paragraph was to the upcoming one. And the current SOTA for TTS includes breathing too, so you can't just put a fixed empty pause between your paragraphs. People are chunking by paragraphs anyways (or even sentences) and it works, but the top commercial models support maintaining a context or passing in the most recently generated text for that reason.