3 ms·
The bottleneck is currently TTS. The best option is probably Eleven Labs, but response times are unpredictable. GPT response times can be worked around by falli
by nojs 4y ago
The bottleneck is currently TTS. The best option is probably Eleven Labs, but response times are unpredictable. GPT response times can be worked around by falling back to a faster model, but you can't do that with TTS because the voice needs to be consistent. It seems like current state of the art are diffusion models ala DALL-E, see e.g. [1] (the developer, James Betker now incidentally works for OpenAI). It's nontrivial to turn this into something that works in real-time without a decent budget, though.
Whisper (for transcription) is insanely fast and good.
1. https://github.com/neonbjb/tortoise-tts https://github.com/neonbjb/tortoise-tts
- ilaksh 4y agoThe Eleven Labs guy told me they put out an update that made it more consistent, which maybe it is slightly better, but still unpredictable like you say. Maybe you could ask them about it again. I did suggest they make a higher queue priority available for higher level paid tiers.