xtts2 with deepspeed and whisper + Ctranslate2 with or without distil-whisper weights already run at many multiples of realtime on GPU. For the top-top end Whi

xtts2 with deepspeed and whisper + Ctranslate2 with or without distil-whisper weights already run at many multiples of realtime on GPU. For the top-top end Whisper Large with distil-whisper and TensorRT-LLM hits at least 50x realtime on an RTX 4090. Note that my application only uses very short speech segments. Longer speech segments increase the realtime multiple SIGNIFICANTLY (as in hitting 150x realtime) due to batching, etc. There’s also Nvidia Canary which is smaller, faster, and more accurate. It’s pretty new and the ecosystem around it is more or less nonexistent but it’s increasingly well supported in Nvidia world at least.