3 ms·
> Limitations >Timestamps/Speaker diarization. The model does not feature either of these. What a shame. Is whisperx still the best choice if you want timesta
by gruez 7mo ago
> Limitations
>Timestamps/Speaker diarization. The model does not feature either of these.
What a shame. Is whisperx still the best choice if you want timestamps/diarization?
- akreal 7mo agoWhisperX is not a model but a software package built around Whisper and some other models, including diarization and alignment ones. Something similar will be built around the Cohere Transcribe model, maybe even just an integration to WhisperX itself.
- GaggiX 7mo agoThere is also: https://github.com/linto-ai/whisper-timestamped https://github.com/linto-ai/whisper-timestamped It doesn't use an extra model (so it supports every language that works with Whisper out of the box and use less memory), it works by applying Dynamic Time Warping to cross-attention weights.
- oezi 7mo agoJust a warning that plain WhisperX is more accurate and Whisper-timestamped has many weird quirks.
- bartman 7mo agoEven in the commercial space, there’s a lack of production grade ASR APIs that support diarization and word level timestamps. My experiences with Google’s Chirp have been horrendous, with it sometimes skipping sections of speech entirely, hallucinating speech where the audio contains noise, and unreliable word level timestamps. And this all is even with using their new audio prefiltering feature. AWS works slightly better, but also has trouble with keeping word level timestamps in sync. Whisper is nice but hallucinates regularly. OpenAI’s new transcription models are delivering accurate output but do not support word level timestamps… A lot of this could be worked around by sending the resulting transcripts through a few layers of post processing, but… I just want to pay for an API that is reliable and saves me from doing all that work.
- stavros 7mo agoIsn't Elevenlabs the best in this?
- bartman 7mo agoI've not tested their speech-to-text yet, but based on the docs it looks promising. Thanks for the suggestion!
- stavros 7mo agoIt's fantastic, and their diarization is spot on as well.
- gardnr 7mo agoThey can have issues with the timestamps: https://github.com/elevenlabs/elevenlabs-python/issues/707 https://github.com/elevenlabs/elevenlabs-python/issues/707
- catlifeonmars 7mo agoI wonder if you could run multiple models and average out the timestamps, kind of like how atomic clocks are used together and not separately
- lifesaverluke 7mo agoFor podcasts there is this https://news.ycombinator.com/item?id=47584376 https://news.ycombinator.com/item?id=47584376
- angel- 7mo agoLink doesn't work for me, can you double check it please? Or tell the name of it so I can look it up? Thanks!
- satvikpendem 7mo agoEnable show dead in your HN profile settings. The link works then as it's a dead show HN post.
- atoav 7mo agoI would try Qwen-ASR: https://qwen.ai/blog?id=qwen3asr https://qwen.ai/blog?id=qwen3asr See the very bottom of the page for a transcription with timestamps.
- stavros 7mo agoDiarization is done separately to ASR anyway (it's usually a separate run, after the ASR).
- mcbetz 7mo agoMistral Voxtral has timestamps and diarization and does a good job for German (have not tested for other languages yet).