3 ms·
Not that I can see, the developer's roadmap[1] currently is at writing a blogpost about it & trying different sampling methods, expanding to the large model loo
by garblegarble 3y ago
Not that I can see, the developer's roadmap[1] currently is at writing a blogpost about it & trying different sampling methods, expanding to the large model looks to be a long way off (which is a real pity, even if there was just a finetune of large for english that would be a big help over the existing small english finetune).
Could you go into more detail about your workflow? I'd been considering a two-pass approach myself until I discovered tinydiarize mentioned in whisper.cpp's --help text
1: https://github.com/akashmjn/tinydiarize#roadmap https://github.com/akashmjn/tinydiarize#roadmap
- 0cf8612b2e1e 3y agoI have not yet done this, but my thinking is that the diarization process would be useful data, but not so much that I would be willing to accept the quality dip of using the tiny trained model. I already run the large model at the word level, so I have high quality transcription + timestamps. Then I would perform the second whisper-pass with the diarization and discard all of the output except when a speaker change is identified. Merge the two results based on timestamps and there you go.
- garblegarble 3y agoInteresting - I'd been considering basically that two-pass approach in reverse (diarize, isolate each utterance in a per-speaker file with a vocal separator between each clip so that I don't have to rely on timestamps being perfectly accurate, run through whisper - could also do a file per utterance, but setup time seems like it would dominate in that case). Concern I had about my approach was that it'd be too different from whisper's training set of conversational data, and that might cause accuracy to take a dive.