3 ms·
Does Whisper do streaming translation? I'm imagining the near future will see a portable fast streaming model for real-time voice translation, piped into text
by taberiand 4y ago
Does Whisper do streaming translation?
I'm imagining the near future will see a portable fast streaming model for real-time voice translation, piped into text to speech. Hook it up to an earpiece and you've got a real-life Babelfish
- speedgoose 4y agoYes it does streaming but you may have to reduce the quality to keep up if your machine is not very fast.
- woodson 4y agoWhisper doesn’t do streaming transcription or translation. It’s one of its architectural disadvantages over more commonly used ASR approaches, such as transducers (see e.g. Microsoft Research’s system trained on ~350k hours that does streaming transcription and translation: https://arxiv.org/abs/2211.02499 https://arxiv.org/abs/2211.02499).