3 ms·
That's the implication. If the distil models are same format as original openai models then the Distil models can be converted for faster-whisper use as per the
by rjwilmsi 3y ago
That's the implication. If the distil models are same format as original openai models then the Distil models can be converted for faster-whisper use as per the conversion instructions on https://github.com/guillaumekln/faster-whisper/ https://github.com/guillaumekln/faster-whisper/
So then we'll see whether we get the 6x model speedup on top of the stated 4x faster-whisper code speedup, at same/nearly same accuracy.
I would generally start with the assumption that if something is significantly faster the accuracy has to suffer a bit, but increasing model size and/or settings such as beam size to compensate should allow same accuracy and higher performance (just not all of the stated performance gain).
- kkielhofner 3y agoJust a point of clarification - faster-whisper references it but ctranslate2[0] is what's really doing the magic here. Ctranslate2 is a sleeper powerhouse project that enables a lot. They should be up front and center and get the credit they deserve. [0] - https://github.com/OpenNMT/CTranslate2 https://github.com/OpenNMT/CTranslate2