4 ms·

As a heavy user of MacWhisper (for dictation), I'm looking forward to better speech-to-text models. MacWhisper with Whisper Large v3 Turbo model works fine, but

by jwr 8mo ago

As a heavy user of MacWhisper (for dictation), I'm looking forward to better speech-to-text models. MacWhisper with Whisper Large v3 Turbo model works fine, but latency adds up quickly, especially if you use online LLMs for post-processing (and it really improves things a lot).

regularfry 8mo ago
If you haven't already, give the models that Handy supports a try. They're not Whisper-large quality, but some of them are very fast.
kavith 8mo ago
Not sure if this will help but I've set up Handy [1] with Parakeet V2 for STT and gpt-oss-120b on Cerebras [2] for post-processing and I'm happy with the performance of this setup! [1] https://handy.computer/ https://handy.computer/ [2] https://www.cerebras.ai/ https://www.cerebras.ai/
- jiehong 8mo ago
  parakeet v3 is also nice, and better for most languages.
  - vunderba 8mo ago
    The latest build of Handy actually supports Parakeet V3 (among other models) under the covers. Agreed that it's a very solid multilingual model. https://github.com/cjpais/Handy https://github.com/cjpais/Handy
- jwr 8mo ago
  But the advantage here is — what? That I don't have to pay for it? I'm glad it works for you, but I'm in a (solo) business setup where I'd much rather pay a (very reasonable) amount of money for a tool that gets things right and smooths out the edges.
kermitime 8mo ago
the parakeet TDT models that are coreml optimized by fluid audio are hands down the fastest local models i’ve tried— worth checking out! (unloading to the NPU is where the edge is) https://huggingface.co/FluidInference/parakeet-tdt-0.6b-v2-coreml https://huggingface.co/FluidInference/parakeet-tdt-0.6b-v2-c... https://github.com/FluidInference/FluidAudio https://github.com/FluidInference/FluidAudio The devs are responsive and active and nice on their discord too. You’ll find discussions on all the latest whizbangs with VAD, TTS, EOU etc
smcleod 8mo ago
Handy with parakeet v2 is excellent
atiorh 8mo ago
MacWhisper supports 10x faster models with the same accuracy like Parakeet v2 (they were the first to do it 6-9 months ago). Have you tried those?
- jwr 8mo ago
  Yes. I found Parakeet V3 (1.24GB) to be worse than Whisper (Large v3 Turbo, 20240930). Subtly worse, but still. I went back to Whisper because with dictation if you can't trust the tool, you won't bother to dictate.
shorggth 7mo ago
In my benchmarks Parakeet v3 actually beats Whisper Turbo on CER (3.8% vs 4.0% on BBC News audio) while being nearly 2x faster (25x vs 14x RT on M1). The accuracy gap may depend on audio type — Parakeet tends to be better on clean speech while Whisper handles noisy/accented audio more robustly.

by jwr 8mo ago

regularfry 8mo ago

If you haven't already, give the models that Handy supports a try. They're not Whisper-large quality, but some of them are very fast.

kavith 8mo ago

Not sure if this will help but I've set up Handy [1] with Parakeet V2 for STT and gpt-oss-120b on Cerebras [2] for post-processing and I'm happy with the performance of this setup! [1] https://handy.computer/ https://handy.computer/ [2] https://www.cerebras.ai/ https://www.cerebras.ai/

jiehong 8mo ago
parakeet v3 is also nice, and better for most languages.
- vunderba 8mo ago
  The latest build of Handy actually supports Parakeet V3 (among other models) under the covers. Agreed that it's a very solid multilingual model. https://github.com/cjpais/Handy https://github.com/cjpais/Handy
jwr 8mo ago
But the advantage here is — what? That I don't have to pay for it? I'm glad it works for you, but I'm in a (solo) business setup where I'd much rather pay a (very reasonable) amount of money for a tool that gets things right and smooths out the edges.

kermitime 8mo ago

the parakeet TDT models that are coreml optimized by fluid audio are hands down the fastest local models i’ve tried— worth checking out! (unloading to the NPU is where the edge is) https://huggingface.co/FluidInference/parakeet-tdt-0.6b-v2-coreml https://huggingface.co/FluidInference/parakeet-tdt-0.6b-v2-c... https://github.com/FluidInference/FluidAudio https://github.com/FluidInference/FluidAudio The devs are responsive and active and nice on their discord too. You’ll find discussions on all the latest whizbangs with VAD, TTS, EOU etc

smcleod 8mo ago

Handy with parakeet v2 is excellent

atiorh 8mo ago

MacWhisper supports 10x faster models with the same accuracy like Parakeet v2 (they were the first to do it 6-9 months ago). Have you tried those?

jwr 8mo ago
Yes. I found Parakeet V3 (1.24GB) to be worse than Whisper (Large v3 Turbo, 20240930). Subtly worse, but still. I went back to Whisper because with dictation if you can't trust the tool, you won't bother to dictate.

shorggth 7mo ago

In my benchmarks Parakeet v3 actually beats Whisper Turbo on CER (3.8% vs 4.0% on BBC News audio) while being nearly 2x faster (25x vs 14x RT on M1). The accuracy gap may depend on audio type — Parakeet tends to be better on clean speech while Whisper handles noisy/accented audio more robustly.