3 ms·
whisper MUST be combined with silence detection / VAD
by wahnfrieden 1y ago
whisper MUST be combined with silence detection / VAD
- pferde 1y agoAh, the good old "you're holding it wrong". What good is a speech recognition tool that literally hears imaginary voices?
- Xmd5a 1y agofaster-whisper has a min_silence_duration_ms option
- wahnfrieden 1y agoThere are much higher quality VAD solutions available
- DANmode 1y agoPlease name a couple to get someone started who's hacking on webapps? I'd really appreciate it.
- DANmode 1y ago(as would future readers, I'm sure)
- DANmode 1y agohttps://github.com/ten-framework/ten-vad https://github.com/ten-framework/ten-vad
- wahnfrieden 1y agoI last used silero but haven’t kept up with stage of the art so didn’t mention it
- deleted 1y ago[deleted]
- zettabomb 1y agoConsidering that if you DO use VAD (voice activity detection), it's the best open weights voice recognition model by a very wide margin, it's quite good. I'd be willing to be that commercial products that "don't have this problem" are using VAD as well, and that this is well known to them. But Whisper is just the weights, and I suppose a simple reference implementation, not a full product.
- bmacho 1y ago> What good is a speech recognition tool that literally hears imaginary voices? Well, if it is supposed to work after silence detection, then it is good for speech recognition I guess. It's like blaming a wheel why is it circular, you can't sit on it. It's a part of a larger machine.
- dumbfounder 1y agoJust lay the wheel on its side and it makes a fine seat.
- wahnfrieden 1y agoYes, you are holding it wrong. The good of it is that it does not output imaginary voices when used with VAD. Show us a technology with better results that does not use VAD. If you can’t, then I’m not sure what you’re arguing against except superficialities so inconsequential that I can’t comprehend the condescension. The results speak for itself
- xandrius 1y agoSo if a tool has a process to have it perform at its best then it's a problem? Do you also moan that before applying glue to a surface or it won't stick? Or if you need to drill a guiding hole before making a larger one in wood? Or that you need to use truly prime numbers for a security key to actually be safe?
- nhecker 1y ago>imaginary voices On the other hand, I can imagine that when things get quiet and the signal-to-noise ratio gets close to zero, random background audio (or randomness introduced in the transcription model) will be enough to tickle a critical number of neurons and elicit hallucinations. The related thought exercise is this: Try scanning across the band with an AM or sideband radio, and after a while your brain will start to wonder "was that a voice I just heard, or music perhaps?" when in reality it was just environmental static.
- DANmode 1y agoWhat's VAD?
- maxbond 1y agoVoice Activity Detection (it predicts whether a short clip contains speech, eg to mute your microphone when you aren't speaking).
- cmiles74 1y agoIf that's truly the case then they should make it part of the product, IMHO.
- wahnfrieden 1y agoHow is it not the case? It is unusable without VAD or editing. I don't understand what you're questioning I agree their products could be better "end to end" integrated. Meanwhile there is a continuously-improving field of work for detecting speech (which Whisper is incapable of). They offer official "cookbooks" with guidance on an approach they recommend: https://cookbook.openai.com/examples/whisper_processing_guide https://cookbook.openai.com/examples/whisper_processing_guid... > At times, files with long silences at the beginning can cause Whisper to transcribe the audio incorrectly. We'll use Pydub to detect and trim the silence. (Official OpenAI quote)
- DANmode 1y agoWhat's a good starter VAD lib, and if you know, the best implementation of something like this to use in a browser-based app? Say if I wanted to use it for Voice Nav, or Voice Input, but not piss off random people speaking the wrong language.