5 ms·
Aren't there text to talk solution that can receive a stream of text so one doesn't have to wait for llama to finish production before getting the answer talked
by avereveard 3y ago
Aren't there text to talk solution that can receive a stream of text so one doesn't have to wait for llama to finish production before getting the answer talked out?
I guess it'd only work if the model can keep the buffer filled fast enough so the tts engine doesn't stall.
- taneq 3y agoJust have llama.cpp emit an “um”, “uhh” etc. when the buffer’s down to one word. :D
- umtksa 3y agoyou laught at the end but I love this solution
- ryanklee 3y agoHumans have loved the same solution since we first started talking, as well
- mirekrusin 3y agoDon’t forget to mix it with “apparently”, “you know what I’m saying”, “I mean”, “you know” etc.
- ryanklee 3y agoI can't tell if you're disparaging the usage or not (truly, I can't tell), but such utterances exist because they serve a real function. Disfluency is an integral part of speech.
- vidarh 3y agoI think it's a good idea, if done well. It could also potentially be combined with dynamically adjusting speed of the speech, and reducing or increasing the use of shortcuts and contractions, making word replacements. I know wish for a model built to be a low-computation filter that takes text in and produces padded text out intended for TTS and annotated with pauses or sounds and extra words that maintains the same meaning but provides the ability to dynamically adjust the level of verbosity to maintain a fixed rate of words per minute.
- taneq 3y agoI always thought of them as the human equivalent of hard drive noises. <brrrrr brrbrrbr>
- taneq 3y agoarray_rand($verbal_fry[$locale]) /* :D */
- HPsquared 3y agoTiming and emphasis work better if you know where the sentence is going; otherwise you sound like one of those translators at the UN with the flat stream of words.
- numpad0 3y agoI suppose it could be buffered, and used to only shorten the wait in cases with long responses, rather than aiming for perfect and push out words as they come. Besides, GPTs sound like translators anyway.
- moffkalast 3y agoI mean.. 99% of the current TTS engines wouldn't know timing and emphasis if it hit them. Besides humans do this all the time, we start saying words before we even have, uhh, any idea how we're gonna end the sentence and for the most part it, uhh, works out. Should be doable.
- unshavedyak 3y agoAlso there's a big difference between waiting for a buffer of outputs and waiting for all the outputs.
- HPsquared 3y agoThe difference is that if you are the one saying the sentence, you at least have some idea where it's going. Receiving one word at a time from a different source isn't the same.
- moffkalast 3y ago> you at least have some idea where it's going Unless you're Michael Scott.
- ccoreilly 3y agoElevenLabs and Gemelo.AI are services that both support text input streaming for exactly this use-case. I am not aware of any open-source Incremental TTS (this is the term used in research afaik) model but you can already achieve somthing similar by buffering tokens and sending them to the TTS model on punctuation characters.
- selfhoster11 3y agoElevenLabs only has streaming output available. I've had a look at both recently and ElevenLabs doesn't have streaming input listed as a feature. Would be cool if it had it, though. You could probably approximate this on a sentence level, but you would need to do some normalisation to make the speech sound even.