3 ms·
Very cool! How is this differentiated from ChatGPT voice?
by codekansas 3y ago
Very cool! How is this differentiated from ChatGPT voice?
- huac 3y agoChatGPT voice takes the cascaded approach - Whisper to transcribe speech to text, then to GPT, then to TTS. We skip the transcription step. Latency: OpenAI's implementation is quite slow - 5+ seconds to get a reply - but even optimized solutions are far slower than this approach. Fundamentally, most transcription models do audio -> audio encoder -> text decoder. By skipping the text decoder, we are always going to win on speed, we just do less work. Emotion: transcription cannot transcribe emotion, only the text, but emotion - eg if the user is sarcastic, angry, happy, etc - plays a big role. Transcription errors: in a cascaded approach, any mistake in an earlier step propagates to the latter steps. Mistranscription means that the LLM will make a mistake. But the LLM's are really really smart, certainly smarter than the transcription models - they'll get it right more often.