11 ms·

Voxtral Transcribe 2

1012 points by meetpateltech 8mo ago

observationist 8mo ago
Native diarization, this looks exciting. edit: or not, no diarization in real-time. https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602 https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-26... ~9GB model.
- coder543 8mo ago
  The diarization is on Voxtral Mini Transcribe V2, not Voxtral Mini 4B.
  - observationist 8mo ago
    Ahh, yeah, and it's explicitly not working for realtime streams. Good catch!
  - sbrother 8mo ago
    Do you have experience with that model for diarization? Does it feel accurate, and what's its realtime factor on a typical GPU? Diarization has been the biggest thorn in my side for a long time..
    - coder543 8mo ago
      > Do you have experience with that model No, I just heard about it this morning.
    - ashenke 8mo ago
      You can test it yourself for free on https://console.mistral.ai/build/audio/speech-to-text https://console.mistral.ai/build/audio/speech-to-text I tried it on an english-speaking podcast episode, and apart from identying one host as two different speakers (but only once for a few sentences at the start), the rest was flawless from what I could see
      - sbrother 8mo ago
        Amazing. Thank you.
serf 8mo ago
things I hate: "Click me to try now!" banners that lead to a warning screen that says "Oh, only paying members, whoops!" So, you don't mean 'try this out', you mean 'buy this product'. Let's not act like it's a free sampler. I can't comment on the model : i'm not giving them money.
- ReadEvalPost 8mo ago
  You can try it on HF: https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtime https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtim...
  - boobsbr 8mo ago
    I'm impressed.
  - deleted 8mo ago
    [deleted]
mdrzn 8mo ago
There's no comparison to Whisper Large v3 or other Whisper models.. Is it better? Worse? Why do they only compare to gpt4o mini transcribe?
- GaggiX 8mo ago
  Gpt4o mini transcribe is better and actually realtime. Whisper is trained to encode the entire audio (or at least 30s chunks) and then decode it.
  - emmettm 8mo ago
    The linked article claims the average word error rate for Voxtral mini v2 is lower than GPT-4o mini transcribe
    - GaggiX 8mo ago
      Gpt4o mini transcribe is better than whisper, the context is the parent comment.
  - mdrzn 8mo ago
    So "gpt4o mini transcribe" is not just whisper v3 under the hood? Btw it's $0.006 / minute For Whisper API online (with v3 large) I've found "$0.00125 per compute second" which is the cheapest absolute I've ever found.
    - GaggiX 8mo ago
      >So it's not just whisper v3 under the hood? Why it should be Whisper v3? They even released an open model: https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602 https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-26...
    - deleted 8mo ago
      [deleted]
    - breisa 8mo ago
      Deepinfra offers Whisper V3 at 0.00045$ / minute of transcribed audio.
varispeed 8mo ago
[flagged]
- Empact 8mo ago
  Many people speak Russian, including many who do not live in Russia, e.g. about 30% of Ukranians. Beyond that, I don't see how we stand to durably reduce military action by making languages mutually unintelligible. https://simple.wikipedia.org/wiki/Russian_language#/media/File:Russian_language_status_and_proficiency_in_the_World.svg https://simple.wikipedia.org/wiki/Russian_language#/media/Fi...
- laffOr 8mo ago
  Don't they have a partnership with the French Armed Forces? I am sure they are interested in automating Russian Audio or Text (-> Russian Text) -> French text.
  - varispeed 8mo ago
    Fair point.
- gostsamo 8mo ago
  They've chosen languages which would help them to cover the highest percentage of human population..
dmix 8mo ago
> At approximately 4% word error rate on FLEURS and $0.003/min Amazons transcription service is $0.024 per minute, pretty big difference https://aws.amazon.com/transcribe/pricing/ https://aws.amazon.com/transcribe/pricing/
- mdrzn 8mo ago
  Is it 0.003 per minute of audio uploaded, or "compute minute"? For example fal.ai has a Whisper API endpoint priced at "$0.00125 per compute second" which (at 10-25x realtime) is EXTREMELY cheaper than all the competitors.
  - Oras 8mo ago
    I think the point is having it for real-time; this is for conversations rather than transcribing audio files.
    - jamilton 8mo ago
      That quote was for the non-realtime model.
  - tgrowazay 8mo ago
    Both AWS and Mistral prices above are per minute of input audio.
    - Curiositry 8mo ago
      If Voxtral can process rapid speech as well as it claims to, an obvious cost optimization would be to speed up normal laconic speech to the maximum speed the model can handle accurately.
  - 85392_school 8mo ago
    It can actually go much lower. Gemini costs around $0.01/hour of transcription last time I checked.
Archelaos 8mo ago
As a rule of thumb for software that I use regularly, it is very useful to consider the costs over a 10-year period in order to compare it with software that I purchase for lifetime to install at home. So that means 1,798.80 $ for the Pro version. What estimates do others use?
antirez 8mo ago
Italian represents, I believe, the most phonetically advanced human language. It has the right compromise among information density, understandability, and ability to speech much faster to compensate the redundancy. It's like if it had error correction built-in. Note that it's not just that it has the lower error rate, but is also underrepresented in most datasets.
- Archelaos 8mo ago
  This is largely due to the fact that modern Italian is a systematised language that emerged from a literary movement (whose most prominent representative is Alessandro Manzoni) to establish a uniform language for the Italian people. At the time of Italian unification in 1861, only about 2.5% of the population could speak this language.
  - gbalduzzi 8mo ago
    The language itself was not invented for the purpose: it was the language spoken in Florence, than adopted by the literary movement and than selected as the national language. It seems like the best tradeoff between information density and understandability actually comes from the deep latin roots of the language
- gbalduzzi 8mo ago
  I was honestly surprised to find it in the first place, because I assumed English to be at first place given the simpler grammar and the huge dataset available. I agree with your belief, other languages have either lower density (e.g. German) or lower understandability (e.g. English)
  - riffraff 8mo ago
    English has a ton of homophones, way more sounds that differ slightly (long/short vowels), and major pronunciation differences across major "official" languages (think Australia/US/Canada/UK). Italian has one official italian (two, if you count IT_ch, but difference is minor), doesn't pay much attention to stress and vowel length, and only has a few "confusable" sounds (gl/l, gn/n, double consonants, stuff you get wrong in primary school). Italian dialects would be a disaster tho :)
simonw 8mo ago
This demo is really impressive: https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtime https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtim... Don't be confused if it says "no microphone", the moment you click the record button it will request browser permission and then start working. I spoke fast and dropped in some jargon and it got it all right - I said this and it transcribed it exactly right, WebAssembly spelling included: > Can you tell me about RSS and Atom and the role of CSP headers in browser security, especially if you're using WebAssembly?
- Oras 8mo ago
  Thank you for the link! Their playground in Mistral does not have a microphone. it just uploads files, which does not demonstrate the speed and accuracy, but the link you shared does. I tried speaking in 2 languages at once, and it picked it up correctly. Truly impressive for real-time.
  - druskacik 8mo ago
    According to the announcement blog Le Chat is powered by the new model as well: https://chat.mistral.ai/chat https://chat.mistral.ai/chat
  - TacticalCoder 8mo ago
    > Truly impressive for real-time. Impressive indeed. Works way better than the speech recognition I first got demo'ed in... 1998? I remember you had to "click" on the mic everytime you wanted to speak and, well, not only the transcription was bad, it was so bad that it'd try to interpret the sound of the click as a word. It was so bad I told several people not to invest in what was back then a national tech darling: https://en.wikipedia.org/wiki/Lernout_%26_Hauspie https://en.wikipedia.org/wiki/Lernout_%26_Hauspie That turned out to be a massive fraud. But ... > I tried speaking in 2 languages at once, and it picked it up correctly. I'm a native french speaker and I tried with a very simple sentence mixing french and english: "Pour un pistolet je prefere un red dot mais pour une carabine je prefere un ACOG" (aka "For a pistol I prefer a red dot but for a carbine I prefer an ACOG") And instead I got this: "Je prépare un redote, mais pour une carabine, je préfère un ACOG." "Je prépare un redote ..." doesn't mean anything and it's not at all what I said. I like it, it's impressive, but literally the first sentence I tried it got the first half entirely wrong.
satvikpendem 8mo ago
Looks like this model doesn't do realtime diarization, what model should I use if I want that? So far I've only seen paid models do diarization well. I heard about Nvidia NeMo but haven't tried that or even where to try it out.
- breisa 8mo ago
  Not sure if its "realtime" but the recently released VibeVoice-ASR from Microsoft does do diarization. https://huggingface.co/microsoft/VibeVoice-ASR https://huggingface.co/microsoft/VibeVoice-ASR
aavci 8mo ago
What's the cheapest device specs that this could realistically run on?
- kamranjon 8mo ago
  I haven't quite figured out if the open weights they released on huggingface amount to being able to run the (realtime) model locally - i hope so though! For the larger model with diarization I don't think they open sourced anything.
  - IanCal 8mo ago
    The HF page suggests yes, with vllm. > We've worked hand-in-hand with the vLLM team to have production-grade support for Voxtral Mini 4B Realtime 2602 with vLLM. Special thanks goes out to Joshua Deng, Yu Luo, Chen Zhang, Nick Hill, Nicolò Lucchesi, Roger Wang, and Cyrus Leung for the amazing work and help on building a production-ready audio streaming and realtime system in vLLM. https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602 https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-26... https://docs.vllm.ai/en/latest/serving/openai_compatible_server/?h=realtime#realtime-api https://docs.vllm.ai/en/latest/serving/openai_compatible_ser...
pietz 8mo ago
Do we know if this is better than Nvidia Parakeet V3? That has been my go-to model locally and it's hard to imagine there's something even better.
- tylergetsay 8mo ago
  I've been using Parakeet V3 locally and totally ancedotaly this feels more accurate but slightly slower
- czottmann 8mo ago
  I liked Parakeet v3 a lot until it started to drop whole sentences, willy-nilly.
  - cypherpunks01 8mo ago
    Yeah, I think the multilingual improvements in V3 caused some kind of regression for English - I've noticed large blocks occasionally dropped as well, so reverted to v2 for my usage. Specifically nvidia/parakeet-tdt-0.6b-v2 vs nvidia/parakeet-tdt-0.6b-v3
  - WXLCKNO 8mo ago
    Oh god am I glad to read this. Thought it was my microphone or something.
  - d4rkp4ttern 8mo ago
    I didn’t see that but I do get a lot of stutters (words or syllables repeated 5+ times), not sure if it’s a model problem or post processing issue in the Handy app.
- whinvik 8mo ago
  Came here to ask the same question!
- m1el 8mo ago
  I've been using nemotron ASR with my own ported inference, and happy about it: https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b https://huggingface.co/nvidia/nemotron-speech-streaming-en-0... https://github.com/m1el/nemotron-asr.cpp https://github.com/m1el/nemotron-asr.cpp https://huggingface.co/m1el/nemotron-speech-streaming-0.6B-gguf https://huggingface.co/m1el/nemotron-speech-streaming-0.6B-g...
boringg 8mo ago
Pseudo related -- am I the only one uncomfortable using my voice with AI for the concern that once it is in the training model it is forever reproducible? As a non-public person it seems like a risk vector (albeit small),
- ffsm8 8mo ago
  It's a real issue, but why do you only see it in ai? It's true for any case where you're speaking into a microphone Depending on the permissions granted to apps on your mobile device, it can even be passively exfiltrated without you ever noticing - and that's ignoring the video clips people take and put online. Like your grandma uploading to Facebook a short moment from a Christmas meet or similar There have already been successful scams - eg calls from "relatives" (AI) calling family members needing money urgently and convincing them to send the money...
  - boringg 8mo ago
    I completely agree - but I think those scams you refer to are less explicit but could potentially be anywhere. AI I am intentionally providing them my voice. I'm not sure that the value to security risk is good.
dumpstate 8mo ago
I'm on voxtral-mini-latest and that's why I started seeing 500s today lol
janalsncm 8mo ago
I noticed that this model is multilingual and understands 14 languages. For many use cases, we probably only need a single language, and the extra 13 are simply adding extra latency. I believe there will be a trend in the coming years of trimming the fat off of these jack of all trades models. https://aclanthology.org/2025.findings-acl.87/ https://aclanthology.org/2025.findings-acl.87/
- decide1000 8mo ago
  I think this model proves it's very efficient and accurate.
  - ethmarks 8mo ago
    But it could potentially be even more efficient if it was single-language.
- popalchemist 8mo ago
  It doesn't make sense to have a language-restricted transcription model because of code switching. People aren't machines, we don't stick to our native languages without failure. Even monolingual people move in and out of their native language when using "borrowed" words/phrases. A single-language model will often fail to deal with that.
  - javier123454321 8mo ago
    yeah, one example I run into is getting my perplexity phone assistant to play a song in spanish. I cannot for the life of me get a model to translate: "Play señorita a mi me gusta su style on spotify" correctly
  - janalsncm 8mo ago
    Everything is a tradeoff, and different use cases require different tradeoffs: Option A: this model Option B: faster model, only 1 language Option C: same size model, only 1 language but higher quality My point is that option A isn’t always best. And on the borrowed words bit, there’s no rule that we cannot add borrowed words into the vocab. But you don’t need the whole language. I know what deja voux means but I don’t speak French.
XCSme 8mo ago
Is it me or error rate of 3% is really high? If you transcribe a minute of conversation, you'll have like 5 words transcribed wrongly. In an hour podcast, that is 300 wrongly transcribed words.
- cootsnuck 8mo ago
  The error rate for human transcription can be as high as 5%.
  - XCSme 8mo ago
    Oh wow, I thought humans are like 0.1% error rate, if they are native speakers and aware of the subject being discussed.
    - zipy124 8mo ago
      I was skepitcal upon hearing the figure but various sources do indeed back it up and [0] is a pretty interesting paper (old but still relevant human transcibers haven't changed in accuracy). [0] https://www.microsoft.com/en-us/research/wp-content/uploads/2017/06/paper-revised2.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...
      - XCSme 8mo ago
        I think it's actually hard to verify how correct a transcription is, at scale. Curious where those error rate numbers come from, because they should test it on people actually doing their job.
    - rhdunn 8mo ago
      It can depend a lot on different factors like: - familiarity with the accent and/or speaker; - speed and style/cadence of the speech; - any other audio that is happening that can muffle or distort the audio; - etc. It can also take multiple passes to get a decent transcription.
siddbudd 8mo ago
Wired advertises this as "Ultra-Fast Translation"[^1]. A bit weird coming from a tech magazine. I hope it's just a "typo". [^1]: https://www.wired.com/story/mistral-voxtral-real-time-ai-translation/ https://www.wired.com/story/mistral-voxtral-real-time-ai-tra...
- bigyabai 8mo ago
  It might be capable of translation; OpenAI Whisper was a transcription model that could do it.
ewuhic 8mo ago
Can it translate in real time?
- unstatusthequo 8mo ago
  Also curious about this. Just need real time German to English. What does this?
- numpad0 8mo ago
  Real time as in at >1x speed? Probably? Real time as in per-word basis? Probably not?
yewenjie 8mo ago
One week ago I was on the hunt for an open source model that can do diatization and I had to literally give up because I could not find any easy to use setup.
- vojto11 8mo ago
  WhisperX ?
- ashenke 8mo ago
  I don't know if that will change, but right now only the Voxtral Mini Transcribe V2 supports diarization and it's not open-weight. The Voxtral Realtime model doesn't support diarization, but is open-weight.
derac 8mo ago
Any chance Voxtral Mini Transcribe 2 will ever be an open model?
- gunalx 8mo ago
  I think this is it. https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602 https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-26...
jszymborski 8mo ago
I'm guessing I won't be able to finetune this until they come out with a HF tranformers model, right?
gwerbret 8mo ago
I really wish those offering speech-to-text models provided transcription benchmarks specific to particular fields of endeavor. I imagine performance would vary wildly when using jargon peculiar to software development, medicine, physics, and law, as compared to everyday speech. Considering that "enterprise" use is often specialized or sub-specialized, it seems like they're leaving money on Dragon's table by not catering to any of those needs.
- deleted 8mo ago
  [deleted]
deleted 8mo ago
[deleted]
deleted 8mo ago
[deleted]
iagooar 8mo ago
In English it is pretty good. But talk to it in Polish, and suddenly it thinks you speak Russian? Ukranian? Belarus? I would understand if an American company launched this, but for a company being so proud about their European roots, I think it should have better support for major European languages. I tried English + Polish: > All right, I'm not really sure if transcribing this makes a lot of sense. Maybe not. A цьому nie mówisz po polsku. A цьому nie mówisz po polsku, nie po ukrańsku.
- tdb7893 8mo ago
  Yeah, it's too bad. Apparently it only performs well in certain languages: "The model is natively multilingual, achieving strong transcription performance in 13 languages, including English, Chinese, Hindi, Spanish, Arabic, French, Portuguese, Russian, German, Japanese, Korean, Italian, and Dutch"
  - ricardonunez 8mo ago
    It did great English and Spanish, it didn't switch to Portuguese, french nor German, maybe struggle with my accent.
    - scotty79 8mo ago
      Try to warn it you are going to switch language to Portugese. Worked for me.
- mystifyingpoi 8mo ago
  TBH ChatGPT does the same, when I mix Polish and English. Generally getting some cyrillic characters and it gets super confused.
- yko 8mo ago
  That's a mix of Polish and Ukrainian in the transcript. Now, if I try speaking Ukrainian, I'm getting transcript in Russian every time. That's upsetting.
jiehong 8mo ago
It’s nice, but the previous version wasn’t actually that great compared to Parakeet for example. We need better independent comparison to see how it performs against the latest Qwen3-ASR, and so on. I can no longer take at face value the cherry picked comparisons of the companies showing off their new models. For now, NVIDIA Parakeet v3 is the best for my use case, and runs very fast on my laptop or my phone.
- archb 8mo ago
  I like Parakeet as well and use it via Handy on Mac. What app are you using on your phone?
  - jiehong 8mo ago
    Spokenly has it on Mac and iOS, in both cases for free when using parakeet
- nodja 8mo ago
  There is https://huggingface.co/spaces/hf-audio/open_asr_leaderboard https://huggingface.co/spaces/hf-audio/open_asr_leaderboard but it hasn't been updated for half a year.
fph 8mo ago
Is there an open source Android keyboard that would support it? Everything I find is based on Whisper, which is from 2022. Ages ago given how fast AI is evolving.
- antirez 8mo ago
  I wish I had a Google Keyboard that could easily run on Whisper Medium. This is already great. But unfortunately would be too much inference cost, incredibly slow. The problem with Whisper is not the inference quality: medium and large are incredible. Is that the base model is not enough, and the only one with fast inference in mobile devices.
  - hephaes7us 8mo ago
    FUTO keyboard is trying to do this. I think they have some kind of distillation of Whisper running on-device.
    - fph 8mo ago
      They are just shipping the same whisper-small that everyone else is using, and did not much to improve their models since release. Other models have been "coming soon" forever. https://keyboard.futo.org/voice-input-models https://keyboard.futo.org/voice-input-models
- fittingopposite 8mo ago
  Have been using https://github.com/notune/android_transcribe_app https://github.com/notune/android_transcribe_app And pretty happy with it. Fully local and fast and accurate
  - fittingopposite 8mo ago
    This uses Parakeet v3 which is a lot lighter but still very good accuracy
  - luplex 8mo ago
blobinabottle 8mo ago
Impressive results, tested on crappy audio files (in french and english)...
yko 8mo ago
Played with the demo a bit. It's really good at English, and detects language change on the fly. Impressive. But whatever I tried, it could not recognise my Ukrainian and would default to Russian in absolutely ridiculous transcription. Other STT models recognise Ukrainian consistently, so I assume there is a lot of Russian in training material, and zero Ukrainian. Made me really sad.
- breisa 8mo ago
  Thats just the result of the model only supporting russian (and 12 other languages) and not urkainian. It maps to the closest words from training data.
numbers 8mo ago
does anyone know if there's any desktop tools I can use this transcription model with? e.g. something where like Wisper Flow/WillowVoice but with custom model selection
- tietjens 8mo ago
  There is Handy, an open source project meant to be a desktop tool, but I haven’t installed it yet to see how you pick your model. Handy – Free open source speech-to-text app https://github.com/cjpais/Handy https://github.com/cjpais/Handy
- k9294 8mo ago
  Try https://ottex.ai/ https://ottex.ai/ I recently added support for Mistral provider, model is actually a very good one, I personally switched to it as my default model. p.s. the app is free for personal user, has support for both local models and BYOK with OpenRouter, Groq, Mistral, Fireworks, and more coming soon.
tallesborges92 8mo ago
I added it to my bot agent,let’s see how it performs
atentaten 8mo ago
Nice. Can this be ran on a mobile device?
ccleve 8mo ago
This looks great, but it's not clear to me how to use it for a practical task. I need to transcribe about 10 years worth of monthly meetings. These are government hearings with a variety of speakers. All the videos are on YouTube. What's the most practical and cost-effective way to get reasonably accurate transcripts?
- jimmy76615 8mo ago
  If they are on Youtube, try Gemini 3 Flash first. Use AI studio, it lets you insert YouTube videos into context.
- IanCal 8mo ago
  If you use something like youtube-dlp you can download the audio from the meetings, and you could try things out in mistrals ai studio. You could use their api (they have this snippet): ```curl -X POST "https://api.mistral.ai/v1/audio/transcriptions https://api.mistral.ai/v1/audio/transcriptions" \ -H "Authorization: Bearer $MISTRAL_API_KEY" \ -F model="voxtral-mini-latest" \ -F file=@"your-file.m4a" \ -F diarize=true \ -F timestamp_granularities="segment"``` In the api it took 18s to do a 20m audio file I had lying around where someone is reviewing a product. There will, I'm sure, be ways of running this locally up and available soon (if they aren't in huggingface right now) but the API is $0.003/min. If it's something like 120 meetings (10 years of monthly ones) then it's roughly $20 if the meetings are 1hr each. Depending on whether they're 1 or 10 hours (or if they're weekly or monthly but 10 parallel sessions or something) then this might be a price you're willing to pay if you get the results back in an afternoon. edit - their realtime model can be run with vllm, the batch model is not open
- isoprophlex 8mo ago
  - get an API key for this service - make sure you have a list of all these YouTube meeting URLs somewhere - ask your preferred coding assistant to write you up a script that downloads the audio for these videos with yt-dlp & calls Mixtrals' API - ???? - profit
- poulpy123 8mo ago
  > 10 years worth of monthly meetings if it's 1 monthly video and thus 120 videos (or so) you could try recall (getrecall.ai not recall.ai that is a similar product with a similar name). They summarize youtube videos, but you get the transcript. AFAIK you cannot batch the processing and you have to add each video one by one, that's why 100 or 200 videos is doable but probably not thousands.
scotty79 8mo ago
Do you know anything better for Polish language, low quality audio than Whisper large-v3 through WhisperX? This combo has almost unbeatable accuracy and it rejects noises in the background really well. It can even reject people talking in the background. The only better thing I've seen is Ursa model from Speechmatics. Not open weights unfortunately.
sgt 8mo ago
What's the best way to train this further on a specific dialect or accent or even terminology?
mnbbrown 8mo ago
Incroyable! Competitive (if not better) than deepgram nova-3, and much better than assembly and elevenlabs in basically all cases on our internal streaming benchmarking. The dataset is ~100 8kHz call recordings with gnarly UK accents (which I consider to be the final boss of english language ASR). It seems like it's SOTA. Where it does fall down seems to be the latency distribution but I'm testing against the API. Running it locally will no doubt improve that?
antirez 8mo ago
Disappointing how this lacks a clear reference implementation, if not mixed at almost yet unreleased VLLM (nightly version) stuff. I'm ok with Open Weights being a form of OSS in the case of models, because frankly I don't believe that, for large LLMs, it is feasible to release the training data, all the orchestration stuff, and so forth. But it can't be: here are the weights, we partnered with VLLM for inference. Come on. Open Weights must mean that you put me in a situation to write an implementation easily for any hardware. p.s. even the demo uses a remote server via websocket.
sbinnee 8mo ago
3 hours for a single request sounds nice to me. Although the graph suggests that it’s not going to perform as good as openai model I have been using, it is open source and surely I will give it a try.
maxdo 8mo ago
https://www.tavus.io/post/sparrow-1-human-level-conversational-timing-in-real-time-voice https://www.tavus.io/post/sparrow-1-human-level-conversation... how does it compare to sparrow-1?
asah 8mo ago
Smells Like Teen Spirit survives another challenge! Voxtral Transcribe 2: Light up our guns, bring your friends, it's fun to lose and to pretend. She's all the more selfish, sure to know how the dirty world. I wasn't what I'd be best before this gift I think best A little girl is always been Always will until again Well, the lights out, it's a stage And we are now entertainers. I'm just stupid and contagious. And we are now entertainers. I'm a lot of, I'm a final. I'm a skater, I'm a freak. Yeah! Hey! Yeah. And I forget just why I taste it Yeah, I guess it makes me smile I found it hard, it's hard to find the well Whatever, never mind Well, the lights out, it's a stage. You and I are now entertainers. I'm just stupid and contagious. You and I are now entertainers. I'm a lot of, I'm a minor. I'm a killer. I'm a beater. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. And I forget just why I taste it Yeah, I guess it makes me smile I found it hard, it's hard to find the well Whatever, never mind I know, I know, I know, I know, I know Well, the lights out, it's a stage. You and I are now entertainers. I'm just stupid and contagious. You and I are now entertainers. I'm a lot of, I'm a minor. I'm a killer. I'm a beater. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. Google/Musixmatch: Load up on guns, bring your friends It's fun to lose and to pretend She's over-bored, and self-assured Oh no, I know a dirty word Hello, hello, hello, how low? Hello, hello, hello, how low? Hello, hello, hello, how low? Hello, hello, hello With the lights out, it's less dangerous Here we are now, entertain us I feel stupid and contagious Here we are now, entertain us A mulatto, an albino A mosquito, my libido, yeah Hey, yey I'm worse at what I do best And for this gift, I feel blessed Our little group has always been And always will until the end Hello, hello, hello, how low? Hello, hello, hello, how low? Hello, hello, hello, how low? Hello, hello, hello With the lights out, it's less dangerous Here we are now, entertain us I feel stupid and contagious Here we are now, entertain us A mulatto, an albino A mosquito, my libido, yeah Hey, yey And I forget just why I taste Oh yeah, I guess it makes me smile I found it hard, it's hard to find Oh well, whatever, never mind Hello, hello, hello, how low? Hello, hello, hello, how low? Hello, hello, hello, how low? Hello, hello, hello With the lights out, it's less dangerous Here we are now, entertain us I feel stupid and contagious Here we are now, entertain us A mulatto, an albino A mosquito, my libido A denial, a denial A denial, a denial A denial, a denial A denial, a denial A denial
- 8mo ago
Rapzid 8mo ago
Has anyone compared to Deepgram Flux yet for realtime?
harry8 8mo ago
What hardware resources are required for what quality/latency? Multiple high end nvidia or can you run it on your phone on an esp32 offline? Or... Seems like fundamental info for any model announcement. Did I just miss it? Does everyone just know except me?
upcoming-sesame 8mo ago
my struggle with VTT is always the accent. it doesn't understand my English too well because of my non native accent
deleted 8mo ago
[deleted]
deleted 8mo ago
[deleted]
mijoharas 8mo ago
Ok, I guess this is the regular time for me to look for a local realtime transcription solution on Linux, and not finding anything good. Maybe this'll get wrapped into a nice tool later. Does anyone have any recommendations?
- rabf 8mo ago
  I made this for myself, might not work on wayland though if thats an issue. https://github.com/rabfulton/Auriscribe https://github.com/rabfulton/Auriscribe
cyp0633 8mo ago
It performs well on Mandarin audio transcription, considering it's an European company. It's weird though that it keeps adding spaces between single Chinese characters, and mixing traditional & simplified characters.
_blackhawk_ 8mo ago
Cannot wait to try it on Spokenly
- k9294 8mo ago
  Hey, I would really appreciate if you will try https://ottex.ai https://ottex.ai I'm working on a Wispr/Spokenly competitor. It's free without any paywalled features, supports local models and bunch of API providers including Mistral. btw, the model name to pick is `voxtral-mini-latest:transcribe`, I personally switched to it as my main default fast model - it's really good.
krick 8mo ago
Is there some well established independent benchmark where I can easily (looking at a couple of graphs) compare all popular (especially self-hosted) transcription models?
- mottiden 8mo ago
  Not that I am aware of unfortunately
Obertr 8mo ago
Very nice! The thing I am missing is turn detection. In real time audio we need the turn detection to understand when AI should speak. Unfortunately this makes it not a complete deepgram replacement yet!
- nostrebored 8mo ago
  Is deepgram really performing better than open source turn detection models for you? In our tests it is not.
  - Obertr 8mo ago
    what is SOTA?
bytesandbits 8mo ago
wow Mistral really cooked
MaxL93 8mo ago
You know what I'd love to have? This running on my Android smartphone. Google's speech services are garbage and they LOVE to cut me off mid-sentence for no reason, well over half the time. It's maddening.
albert_e 8mo ago
I cant wait for models to get smaller enough that they can run on commodity devices. Hope we can build an app like Whispr Flow using this with the model running completely on device.
owenbrown 8mo ago
The other demos didn't work for me, so I made https://github.com/owenbrown/transcribe https://github.com/owenbrown/transcribe It's just a python script to test the streaming. Wow, Voxtral is amazing. It will be great when someone stitches this up so an LLM starts thinking, researching for you, before you actually finish talking. Like, create a conversation partner with sub 0.5 second latency. For example, you ask it a multi part questions and, as soon as you finish talking, it gives you the answer to the first part while it looks up the rest of the answer, then stitches it together so that there's no break. The 2-3 second latency of existing voice chatbots is a non-started for most humans.
- jwblackwell 8mo ago
  Nice! works well - I couldn't get huggingface to work either
- jpeeler 8mo ago
  Yes, appreciate this. I noticed that with both models voxtral-mini-transcribe-realtime-2602 and voxtral-mini-2602 filler words are ignored. I'd like to be able to count words/sounds, specifically "um" or "uh" for improvement purposes. Any good models that handle that?
barrell 8mo ago
Very happy with all the mistral work. I feel like I'm always one release behind theirs. Last time they released Mistral 3 I commented saying how excited I was to try it out [1] Well, I'm happy to report I integrated the new Mistral 3 and have been truly astounded by the results. I still am not a big fan of the model wrt factual information - it seems to be especially confident and especially wrong if left to it's own devices - but with http://phrasing.app http://phrasing.app I do most of the data aggregation myself and just use an LLM to format it. Mistral 3 was a drop-in replacement for 3x the quality (it was already very very good), 0% error rate for my use case (I had an issue for it occasionally going off the rails that was entirely solved), and sticks to my formatting guidelines perfectly (which even gpt-5-pro failed on). Plus it was somehow even cheaper. I'm using Scribe v2 at the moment for TTS, but I'm very excited now to try integrating Voxtral Transcribe. The language support is a little lacking for my use cases, but I can always fall back to Scribe and amatorize the cost across languages. I actually was due to work on the transcription of phrasing very soon so I guess look forward to my (hopefully) glowing review on their next hn launch! XD [1] https://news.ycombinator.com/item?id=46121889#46122612 https://news.ycombinator.com/item?id=46121889#46122612
qwertytyyuu 8mo ago
This exciting, especially after 11 labs very expensive model
ashu1461 8mo ago
Wondering if most of the AI agents use real time apis or transcription apis.. anyone had experience with building voice agents can comment ?
kranke155 8mo ago
Really cool.
RT_max 8mo ago
[dead]

1012 points by meetpateltech 8mo ago

Native diarization, this looks exciting. edit: or not, no diarization in real-time. https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602 https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-26... ~9GB model.

coder543 8mo ago
The diarization is on Voxtral Mini Transcribe V2, not Voxtral Mini 4B.
- observationist 8mo ago
  Ahh, yeah, and it's explicitly not working for realtime streams. Good catch!
- sbrother 8mo ago
  Do you have experience with that model for diarization? Does it feel accurate, and what's its realtime factor on a typical GPU? Diarization has been the biggest thorn in my side for a long time..
  - coder543 8mo ago
    > Do you have experience with that model No, I just heard about it this morning.
  - ashenke 8mo ago
    You can test it yourself for free on https://console.mistral.ai/build/audio/speech-to-text https://console.mistral.ai/build/audio/speech-to-text I tried it on an english-speaking podcast episode, and apart from identying one host as two different speakers (but only once for a few sentences at the start), the rest was flawless from what I could see
    - sbrother 8mo ago
      Amazing. Thank you.

serf 8mo ago

things I hate: "Click me to try now!" banners that lead to a warning screen that says "Oh, only paying members, whoops!" So, you don't mean 'try this out', you mean 'buy this product'. Let's not act like it's a free sampler. I can't comment on the model : i'm not giving them money.

ReadEvalPost 8mo ago
You can try it on HF: https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtime https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtim...
- boobsbr 8mo ago
  I'm impressed.
- deleted 8mo ago
  [deleted]

mdrzn 8mo ago

There's no comparison to Whisper Large v3 or other Whisper models.. Is it better? Worse? Why do they only compare to gpt4o mini transcribe?

GaggiX 8mo ago
Gpt4o mini transcribe is better and actually realtime. Whisper is trained to encode the entire audio (or at least 30s chunks) and then decode it.
- emmettm 8mo ago
  The linked article claims the average word error rate for Voxtral mini v2 is lower than GPT-4o mini transcribe
  - GaggiX 8mo ago
    Gpt4o mini transcribe is better than whisper, the context is the parent comment.
- mdrzn 8mo ago
  So "gpt4o mini transcribe" is not just whisper v3 under the hood? Btw it's $0.006 / minute For Whisper API online (with v3 large) I've found "$0.00125 per compute second" which is the cheapest absolute I've ever found.
  - GaggiX 8mo ago
    >So it's not just whisper v3 under the hood? Why it should be Whisper v3? They even released an open model: https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602 https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-26...
  - deleted 8mo ago
    [deleted]
  - breisa 8mo ago
    Deepinfra offers Whisper V3 at 0.00045$ / minute of transcribed audio.

varispeed 8mo ago

[flagged]

Empact 8mo ago
Many people speak Russian, including many who do not live in Russia, e.g. about 30% of Ukranians. Beyond that, I don't see how we stand to durably reduce military action by making languages mutually unintelligible. https://simple.wikipedia.org/wiki/Russian_language#/media/File:Russian_language_status_and_proficiency_in_the_World.svg https://simple.wikipedia.org/wiki/Russian_language#/media/Fi...
laffOr 8mo ago
Don't they have a partnership with the French Armed Forces? I am sure they are interested in automating Russian Audio or Text (-> Russian Text) -> French text.
- varispeed 8mo ago
  Fair point.
gostsamo 8mo ago
They've chosen languages which would help them to cover the highest percentage of human population..

dmix 8mo ago

> At approximately 4% word error rate on FLEURS and $0.003/min Amazons transcription service is $0.024 per minute, pretty big difference https://aws.amazon.com/transcribe/pricing/ https://aws.amazon.com/transcribe/pricing/

mdrzn 8mo ago
Is it 0.003 per minute of audio uploaded, or "compute minute"? For example fal.ai has a Whisper API endpoint priced at "$0.00125 per compute second" which (at 10-25x realtime) is EXTREMELY cheaper than all the competitors.
- Oras 8mo ago
  I think the point is having it for real-time; this is for conversations rather than transcribing audio files.
  - jamilton 8mo ago
    That quote was for the non-realtime model.
- tgrowazay 8mo ago
  Both AWS and Mistral prices above are per minute of input audio.
  - Curiositry 8mo ago
    If Voxtral can process rapid speech as well as it claims to, an obvious cost optimization would be to speed up normal laconic speech to the maximum speed the model can handle accurately.
- 85392_school 8mo ago
  It can actually go much lower. Gemini costs around $0.01/hour of transcription last time I checked.

Archelaos 8mo ago

As a rule of thumb for software that I use regularly, it is very useful to consider the costs over a 10-year period in order to compare it with software that I purchase for lifetime to install at home. So that means 1,798.80 $ for the Pro version. What estimates do others use?

antirez 8mo ago

Italian represents, I believe, the most phonetically advanced human language. It has the right compromise among information density, understandability, and ability to speech much faster to compensate the redundancy. It's like if it had error correction built-in. Note that it's not just that it has the lower error rate, but is also underrepresented in most datasets.

Archelaos 8mo ago
This is largely due to the fact that modern Italian is a systematised language that emerged from a literary movement (whose most prominent representative is Alessandro Manzoni) to establish a uniform language for the Italian people. At the time of Italian unification in 1861, only about 2.5% of the population could speak this language.
- gbalduzzi 8mo ago
  The language itself was not invented for the purpose: it was the language spoken in Florence, than adopted by the literary movement and than selected as the national language. It seems like the best tradeoff between information density and understandability actually comes from the deep latin roots of the language
gbalduzzi 8mo ago
I was honestly surprised to find it in the first place, because I assumed English to be at first place given the simpler grammar and the huge dataset available. I agree with your belief, other languages have either lower density (e.g. German) or lower understandability (e.g. English)
- riffraff 8mo ago
  English has a ton of homophones, way more sounds that differ slightly (long/short vowels), and major pronunciation differences across major "official" languages (think Australia/US/Canada/UK). Italian has one official italian (two, if you count IT_ch, but difference is minor), doesn't pay much attention to stress and vowel length, and only has a few "confusable" sounds (gl/l, gn/n, double consonants, stuff you get wrong in primary school). Italian dialects would be a disaster tho :)

simonw 8mo ago

This demo is really impressive: https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtime https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtim... Don't be confused if it says "no microphone", the moment you click the record button it will request browser permission and then start working. I spoke fast and dropped in some jargon and it got it all right - I said this and it transcribed it exactly right, WebAssembly spelling included: > Can you tell me about RSS and Atom and the role of CSP headers in browser security, especially if you're using WebAssembly?

Oras 8mo ago
Thank you for the link! Their playground in Mistral does not have a microphone. it just uploads files, which does not demonstrate the speed and accuracy, but the link you shared does. I tried speaking in 2 languages at once, and it picked it up correctly. Truly impressive for real-time.
- druskacik 8mo ago
  According to the announcement blog Le Chat is powered by the new model as well: https://chat.mistral.ai/chat https://chat.mistral.ai/chat
- TacticalCoder 8mo ago
  > Truly impressive for real-time. Impressive indeed. Works way better than the speech recognition I first got demo'ed in... 1998? I remember you had to "click" on the mic everytime you wanted to speak and, well, not only the transcription was bad, it was so bad that it'd try to interpret the sound of the click as a word. It was so bad I told several people not to invest in what was back then a national tech darling: https://en.wikipedia.org/wiki/Lernout_%26_Hauspie https://en.wikipedia.org/wiki/Lernout_%26_Hauspie That turned out to be a massive fraud. But ... > I tried speaking in 2 languages at once, and it picked it up correctly. I'm a native french speaker and I tried with a very simple sentence mixing french and english: "Pour un pistolet je prefere un red dot mais pour une carabine je prefere un ACOG" (aka "For a pistol I prefer a red dot but for a carbine I prefer an ACOG") And instead I got this: "Je prépare un redote, mais pour une carabine, je préfère un ACOG." "Je prépare un redote ..." doesn't mean anything and it's not at all what I said. I like it, it's impressive, but literally the first sentence I tried it got the first half entirely wrong.

satvikpendem 8mo ago

Looks like this model doesn't do realtime diarization, what model should I use if I want that? So far I've only seen paid models do diarization well. I heard about Nvidia NeMo but haven't tried that or even where to try it out.

breisa 8mo ago
Not sure if its "realtime" but the recently released VibeVoice-ASR from Microsoft does do diarization. https://huggingface.co/microsoft/VibeVoice-ASR https://huggingface.co/microsoft/VibeVoice-ASR

aavci 8mo ago

What's the cheapest device specs that this could realistically run on?

kamranjon 8mo ago
I haven't quite figured out if the open weights they released on huggingface amount to being able to run the (realtime) model locally - i hope so though! For the larger model with diarization I don't think they open sourced anything.
- IanCal 8mo ago
  The HF page suggests yes, with vllm. > We've worked hand-in-hand with the vLLM team to have production-grade support for Voxtral Mini 4B Realtime 2602 with vLLM. Special thanks goes out to Joshua Deng, Yu Luo, Chen Zhang, Nick Hill, Nicolò Lucchesi, Roger Wang, and Cyrus Leung for the amazing work and help on building a production-ready audio streaming and realtime system in vLLM. https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602 https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-26... https://docs.vllm.ai/en/latest/serving/openai_compatible_server/?h=realtime#realtime-api https://docs.vllm.ai/en/latest/serving/openai_compatible_ser...

pietz 8mo ago

Do we know if this is better than Nvidia Parakeet V3? That has been my go-to model locally and it's hard to imagine there's something even better.

tylergetsay 8mo ago
I've been using Parakeet V3 locally and totally ancedotaly this feels more accurate but slightly slower
czottmann 8mo ago
I liked Parakeet v3 a lot until it started to drop whole sentences, willy-nilly.
- cypherpunks01 8mo ago
  Yeah, I think the multilingual improvements in V3 caused some kind of regression for English - I've noticed large blocks occasionally dropped as well, so reverted to v2 for my usage. Specifically nvidia/parakeet-tdt-0.6b-v2 vs nvidia/parakeet-tdt-0.6b-v3
- WXLCKNO 8mo ago
  Oh god am I glad to read this. Thought it was my microphone or something.
- d4rkp4ttern 8mo ago
  I didn’t see that but I do get a lot of stutters (words or syllables repeated 5+ times), not sure if it’s a model problem or post processing issue in the Handy app.
whinvik 8mo ago
Came here to ask the same question!
m1el 8mo ago
I've been using nemotron ASR with my own ported inference, and happy about it: https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b https://huggingface.co/nvidia/nemotron-speech-streaming-en-0... https://github.com/m1el/nemotron-asr.cpp https://github.com/m1el/nemotron-asr.cpp https://huggingface.co/m1el/nemotron-speech-streaming-0.6B-gguf https://huggingface.co/m1el/nemotron-speech-streaming-0.6B-g...

boringg 8mo ago

Pseudo related -- am I the only one uncomfortable using my voice with AI for the concern that once it is in the training model it is forever reproducible? As a non-public person it seems like a risk vector (albeit small),

ffsm8 8mo ago
It's a real issue, but why do you only see it in ai? It's true for any case where you're speaking into a microphone Depending on the permissions granted to apps on your mobile device, it can even be passively exfiltrated without you ever noticing - and that's ignoring the video clips people take and put online. Like your grandma uploading to Facebook a short moment from a Christmas meet or similar There have already been successful scams - eg calls from "relatives" (AI) calling family members needing money urgently and convincing them to send the money...
- boringg 8mo ago
  I completely agree - but I think those scams you refer to are less explicit but could potentially be anywhere. AI I am intentionally providing them my voice. I'm not sure that the value to security risk is good.

dumpstate 8mo ago

I'm on voxtral-mini-latest and that's why I started seeing 500s today lol

janalsncm 8mo ago

I noticed that this model is multilingual and understands 14 languages. For many use cases, we probably only need a single language, and the extra 13 are simply adding extra latency. I believe there will be a trend in the coming years of trimming the fat off of these jack of all trades models. https://aclanthology.org/2025.findings-acl.87/ https://aclanthology.org/2025.findings-acl.87/

decide1000 8mo ago
I think this model proves it's very efficient and accurate.
- ethmarks 8mo ago
  But it could potentially be even more efficient if it was single-language.
popalchemist 8mo ago
It doesn't make sense to have a language-restricted transcription model because of code switching. People aren't machines, we don't stick to our native languages without failure. Even monolingual people move in and out of their native language when using "borrowed" words/phrases. A single-language model will often fail to deal with that.
- javier123454321 8mo ago
  yeah, one example I run into is getting my perplexity phone assistant to play a song in spanish. I cannot for the life of me get a model to translate: "Play señorita a mi me gusta su style on spotify" correctly
- janalsncm 8mo ago
  Everything is a tradeoff, and different use cases require different tradeoffs: Option A: this model Option B: faster model, only 1 language Option C: same size model, only 1 language but higher quality My point is that option A isn’t always best. And on the borrowed words bit, there’s no rule that we cannot add borrowed words into the vocab. But you don’t need the whole language. I know what deja voux means but I don’t speak French.

XCSme 8mo ago

Is it me or error rate of 3% is really high? If you transcribe a minute of conversation, you'll have like 5 words transcribed wrongly. In an hour podcast, that is 300 wrongly transcribed words.

cootsnuck 8mo ago
The error rate for human transcription can be as high as 5%.
- XCSme 8mo ago
  Oh wow, I thought humans are like 0.1% error rate, if they are native speakers and aware of the subject being discussed.
  - zipy124 8mo ago
    I was skepitcal upon hearing the figure but various sources do indeed back it up and [0] is a pretty interesting paper (old but still relevant human transcibers haven't changed in accuracy). [0] https://www.microsoft.com/en-us/research/wp-content/uploads/2017/06/paper-revised2.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...
    - XCSme 8mo ago
      I think it's actually hard to verify how correct a transcription is, at scale. Curious where those error rate numbers come from, because they should test it on people actually doing their job.
  - rhdunn 8mo ago
    It can depend a lot on different factors like: - familiarity with the accent and/or speaker; - speed and style/cadence of the speech; - any other audio that is happening that can muffle or distort the audio; - etc. It can also take multiple passes to get a decent transcription.

siddbudd 8mo ago

Wired advertises this as "Ultra-Fast Translation"[^1]. A bit weird coming from a tech magazine. I hope it's just a "typo". [^1]: https://www.wired.com/story/mistral-voxtral-real-time-ai-translation/ https://www.wired.com/story/mistral-voxtral-real-time-ai-tra...

bigyabai 8mo ago
It might be capable of translation; OpenAI Whisper was a transcription model that could do it.

ewuhic 8mo ago

Can it translate in real time?

unstatusthequo 8mo ago
Also curious about this. Just need real time German to English. What does this?
numpad0 8mo ago
Real time as in at >1x speed? Probably? Real time as in per-word basis? Probably not?

yewenjie 8mo ago

One week ago I was on the hunt for an open source model that can do diatization and I had to literally give up because I could not find any easy to use setup.

vojto11 8mo ago
WhisperX ?
ashenke 8mo ago
I don't know if that will change, but right now only the Voxtral Mini Transcribe V2 supports diarization and it's not open-weight. The Voxtral Realtime model doesn't support diarization, but is open-weight.

derac 8mo ago

Any chance Voxtral Mini Transcribe 2 will ever be an open model?

gunalx 8mo ago
I think this is it. https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602 https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-26...

jszymborski 8mo ago

I'm guessing I won't be able to finetune this until they come out with a HF tranformers model, right?

gwerbret 8mo ago

I really wish those offering speech-to-text models provided transcription benchmarks specific to particular fields of endeavor. I imagine performance would vary wildly when using jargon peculiar to software development, medicine, physics, and law, as compared to everyday speech. Considering that "enterprise" use is often specialized or sub-specialized, it seems like they're leaving money on Dragon's table by not catering to any of those needs.

deleted 8mo ago
[deleted]

deleted 8mo ago

[deleted]

deleted 8mo ago

[deleted]

iagooar 8mo ago

In English it is pretty good. But talk to it in Polish, and suddenly it thinks you speak Russian? Ukranian? Belarus? I would understand if an American company launched this, but for a company being so proud about their European roots, I think it should have better support for major European languages. I tried English + Polish: > All right, I'm not really sure if transcribing this makes a lot of sense. Maybe not. A цьому nie mówisz po polsku. A цьому nie mówisz po polsku, nie po ukrańsku.

tdb7893 8mo ago
Yeah, it's too bad. Apparently it only performs well in certain languages: "The model is natively multilingual, achieving strong transcription performance in 13 languages, including English, Chinese, Hindi, Spanish, Arabic, French, Portuguese, Russian, German, Japanese, Korean, Italian, and Dutch"
- ricardonunez 8mo ago
  It did great English and Spanish, it didn't switch to Portuguese, french nor German, maybe struggle with my accent.
  - scotty79 8mo ago
    Try to warn it you are going to switch language to Portugese. Worked for me.
mystifyingpoi 8mo ago
TBH ChatGPT does the same, when I mix Polish and English. Generally getting some cyrillic characters and it gets super confused.
yko 8mo ago
That's a mix of Polish and Ukrainian in the transcript. Now, if I try speaking Ukrainian, I'm getting transcript in Russian every time. That's upsetting.

jiehong 8mo ago

It’s nice, but the previous version wasn’t actually that great compared to Parakeet for example. We need better independent comparison to see how it performs against the latest Qwen3-ASR, and so on. I can no longer take at face value the cherry picked comparisons of the companies showing off their new models. For now, NVIDIA Parakeet v3 is the best for my use case, and runs very fast on my laptop or my phone.

archb 8mo ago
I like Parakeet as well and use it via Handy on Mac. What app are you using on your phone?
- jiehong 8mo ago
  Spokenly has it on Mac and iOS, in both cases for free when using parakeet
nodja 8mo ago
There is https://huggingface.co/spaces/hf-audio/open_asr_leaderboard https://huggingface.co/spaces/hf-audio/open_asr_leaderboard but it hasn't been updated for half a year.

fph 8mo ago

Is there an open source Android keyboard that would support it? Everything I find is based on Whisper, which is from 2022. Ages ago given how fast AI is evolving.

antirez 8mo ago
I wish I had a Google Keyboard that could easily run on Whisper Medium. This is already great. But unfortunately would be too much inference cost, incredibly slow. The problem with Whisper is not the inference quality: medium and large are incredible. Is that the base model is not enough, and the only one with fast inference in mobile devices.
- hephaes7us 8mo ago
  FUTO keyboard is trying to do this. I think they have some kind of distillation of Whisper running on-device.
  - fph 8mo ago
    They are just shipping the same whisper-small that everyone else is using, and did not much to improve their models since release. Other models have been "coming soon" forever. https://keyboard.futo.org/voice-input-models https://keyboard.futo.org/voice-input-models
fittingopposite 8mo ago
Have been using https://github.com/notune/android_transcribe_app https://github.com/notune/android_transcribe_app And pretty happy with it. Fully local and fast and accurate
- fittingopposite 8mo ago
  This uses Parakeet v3 which is a lot lighter but still very good accuracy
- luplex 8mo ago

blobinabottle 8mo ago

Impressive results, tested on crappy audio files (in french and english)...

yko 8mo ago

Played with the demo a bit. It's really good at English, and detects language change on the fly. Impressive. But whatever I tried, it could not recognise my Ukrainian and would default to Russian in absolutely ridiculous transcription. Other STT models recognise Ukrainian consistently, so I assume there is a lot of Russian in training material, and zero Ukrainian. Made me really sad.

breisa 8mo ago
Thats just the result of the model only supporting russian (and 12 other languages) and not urkainian. It maps to the closest words from training data.

numbers 8mo ago

does anyone know if there's any desktop tools I can use this transcription model with? e.g. something where like Wisper Flow/WillowVoice but with custom model selection

tietjens 8mo ago
There is Handy, an open source project meant to be a desktop tool, but I haven’t installed it yet to see how you pick your model. Handy – Free open source speech-to-text app https://github.com/cjpais/Handy https://github.com/cjpais/Handy
k9294 8mo ago
Try https://ottex.ai/ https://ottex.ai/ I recently added support for Mistral provider, model is actually a very good one, I personally switched to it as my default model. p.s. the app is free for personal user, has support for both local models and BYOK with OpenRouter, Groq, Mistral, Fireworks, and more coming soon.

tallesborges92 8mo ago

I added it to my bot agent,let’s see how it performs

atentaten 8mo ago

Nice. Can this be ran on a mobile device?

ccleve 8mo ago

This looks great, but it's not clear to me how to use it for a practical task. I need to transcribe about 10 years worth of monthly meetings. These are government hearings with a variety of speakers. All the videos are on YouTube. What's the most practical and cost-effective way to get reasonably accurate transcripts?

jimmy76615 8mo ago
If they are on Youtube, try Gemini 3 Flash first. Use AI studio, it lets you insert YouTube videos into context.
IanCal 8mo ago
If you use something like youtube-dlp you can download the audio from the meetings, and you could try things out in mistrals ai studio. You could use their api (they have this snippet): ```curl -X POST "https://api.mistral.ai/v1/audio/transcriptions https://api.mistral.ai/v1/audio/transcriptions" \ -H "Authorization: Bearer $MISTRAL_API_KEY" \ -F model="voxtral-mini-latest" \ -F file=@"your-file.m4a" \ -F diarize=true \ -F timestamp_granularities="segment"``` In the api it took 18s to do a 20m audio file I had lying around where someone is reviewing a product. There will, I'm sure, be ways of running this locally up and available soon (if they aren't in huggingface right now) but the API is $0.003/min. If it's something like 120 meetings (10 years of monthly ones) then it's roughly $20 if the meetings are 1hr each. Depending on whether they're 1 or 10 hours (or if they're weekly or monthly but 10 parallel sessions or something) then this might be a price you're willing to pay if you get the results back in an afternoon. edit - their realtime model can be run with vllm, the batch model is not open
isoprophlex 8mo ago
- get an API key for this service - make sure you have a list of all these YouTube meeting URLs somewhere - ask your preferred coding assistant to write you up a script that downloads the audio for these videos with yt-dlp & calls Mixtrals' API - ???? - profit
poulpy123 8mo ago
> 10 years worth of monthly meetings if it's 1 monthly video and thus 120 videos (or so) you could try recall (getrecall.ai not recall.ai that is a similar product with a similar name). They summarize youtube videos, but you get the transcript. AFAIK you cannot batch the processing and you have to add each video one by one, that's why 100 or 200 videos is doable but probably not thousands.

scotty79 8mo ago

Do you know anything better for Polish language, low quality audio than Whisper large-v3 through WhisperX? This combo has almost unbeatable accuracy and it rejects noises in the background really well. It can even reject people talking in the background. The only better thing I've seen is Ursa model from Speechmatics. Not open weights unfortunately.

sgt 8mo ago

What's the best way to train this further on a specific dialect or accent or even terminology?

mnbbrown 8mo ago

Incroyable! Competitive (if not better) than deepgram nova-3, and much better than assembly and elevenlabs in basically all cases on our internal streaming benchmarking. The dataset is ~100 8kHz call recordings with gnarly UK accents (which I consider to be the final boss of english language ASR). It seems like it's SOTA. Where it does fall down seems to be the latency distribution but I'm testing against the API. Running it locally will no doubt improve that?

antirez 8mo ago

Disappointing how this lacks a clear reference implementation, if not mixed at almost yet unreleased VLLM (nightly version) stuff. I'm ok with Open Weights being a form of OSS in the case of models, because frankly I don't believe that, for large LLMs, it is feasible to release the training data, all the orchestration stuff, and so forth. But it can't be: here are the weights, we partnered with VLLM for inference. Come on. Open Weights must mean that you put me in a situation to write an implementation easily for any hardware. p.s. even the demo uses a remote server via websocket.

sbinnee 8mo ago

3 hours for a single request sounds nice to me. Although the graph suggests that it’s not going to perform as good as openai model I have been using, it is open source and surely I will give it a try.

maxdo 8mo ago

https://www.tavus.io/post/sparrow-1-human-level-conversational-timing-in-real-time-voice https://www.tavus.io/post/sparrow-1-human-level-conversation... how does it compare to sparrow-1?

asah 8mo ago

Smells Like Teen Spirit survives another challenge! Voxtral Transcribe 2: Light up our guns, bring your friends, it's fun to lose and to pretend. She's all the more selfish, sure to know how the dirty world. I wasn't what I'd be best before this gift I think best A little girl is always been Always will until again Well, the lights out, it's a stage And we are now entertainers. I'm just stupid and contagious. And we are now entertainers. I'm a lot of, I'm a final. I'm a skater, I'm a freak. Yeah! Hey! Yeah. And I forget just why I taste it Yeah, I guess it makes me smile I found it hard, it's hard to find the well Whatever, never mind Well, the lights out, it's a stage. You and I are now entertainers. I'm just stupid and contagious. You and I are now entertainers. I'm a lot of, I'm a minor. I'm a killer. I'm a beater. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. And I forget just why I taste it Yeah, I guess it makes me smile I found it hard, it's hard to find the well Whatever, never mind I know, I know, I know, I know, I know Well, the lights out, it's a stage. You and I are now entertainers. I'm just stupid and contagious. You and I are now entertainers. I'm a lot of, I'm a minor. I'm a killer. I'm a beater. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. I'm a nerd. Google/Musixmatch: Load up on guns, bring your friends It's fun to lose and to pretend She's over-bored, and self-assured Oh no, I know a dirty word Hello, hello, hello, how low? Hello, hello, hello, how low? Hello, hello, hello, how low? Hello, hello, hello With the lights out, it's less dangerous Here we are now, entertain us I feel stupid and contagious Here we are now, entertain us A mulatto, an albino A mosquito, my libido, yeah Hey, yey I'm worse at what I do best And for this gift, I feel blessed Our little group has always been And always will until the end Hello, hello, hello, how low? Hello, hello, hello, how low? Hello, hello, hello, how low? Hello, hello, hello With the lights out, it's less dangerous Here we are now, entertain us I feel stupid and contagious Here we are now, entertain us A mulatto, an albino A mosquito, my libido, yeah Hey, yey And I forget just why I taste Oh yeah, I guess it makes me smile I found it hard, it's hard to find Oh well, whatever, never mind Hello, hello, hello, how low? Hello, hello, hello, how low? Hello, hello, hello, how low? Hello, hello, hello With the lights out, it's less dangerous Here we are now, entertain us I feel stupid and contagious Here we are now, entertain us A mulatto, an albino A mosquito, my libido A denial, a denial A denial, a denial A denial, a denial A denial, a denial A denial

8mo ago

Rapzid 8mo ago

Has anyone compared to Deepgram Flux yet for realtime?

harry8 8mo ago

What hardware resources are required for what quality/latency? Multiple high end nvidia or can you run it on your phone on an esp32 offline? Or... Seems like fundamental info for any model announcement. Did I just miss it? Does everyone just know except me?

upcoming-sesame 8mo ago

my struggle with VTT is always the accent. it doesn't understand my English too well because of my non native accent

deleted 8mo ago

[deleted]

deleted 8mo ago

[deleted]

mijoharas 8mo ago

Ok, I guess this is the regular time for me to look for a local realtime transcription solution on Linux, and not finding anything good. Maybe this'll get wrapped into a nice tool later. Does anyone have any recommendations?

rabf 8mo ago
I made this for myself, might not work on wayland though if thats an issue. https://github.com/rabfulton/Auriscribe https://github.com/rabfulton/Auriscribe

cyp0633 8mo ago

It performs well on Mandarin audio transcription, considering it's an European company. It's weird though that it keeps adding spaces between single Chinese characters, and mixing traditional & simplified characters.

_blackhawk_ 8mo ago

Cannot wait to try it on Spokenly

k9294 8mo ago
Hey, I would really appreciate if you will try https://ottex.ai https://ottex.ai I'm working on a Wispr/Spokenly competitor. It's free without any paywalled features, supports local models and bunch of API providers including Mistral. btw, the model name to pick is `voxtral-mini-latest:transcribe`, I personally switched to it as my main default fast model - it's really good.

krick 8mo ago

Is there some well established independent benchmark where I can easily (looking at a couple of graphs) compare all popular (especially self-hosted) transcription models?

mottiden 8mo ago
Not that I am aware of unfortunately

Obertr 8mo ago

Very nice! The thing I am missing is turn detection. In real time audio we need the turn detection to understand when AI should speak. Unfortunately this makes it not a complete deepgram replacement yet!

nostrebored 8mo ago
Is deepgram really performing better than open source turn detection models for you? In our tests it is not.
- Obertr 8mo ago
  what is SOTA?

bytesandbits 8mo ago

wow Mistral really cooked

MaxL93 8mo ago

You know what I'd love to have? This running on my Android smartphone. Google's speech services are garbage and they LOVE to cut me off mid-sentence for no reason, well over half the time. It's maddening.

albert_e 8mo ago

I cant wait for models to get smaller enough that they can run on commodity devices. Hope we can build an app like Whispr Flow using this with the model running completely on device.

owenbrown 8mo ago

The other demos didn't work for me, so I made https://github.com/owenbrown/transcribe https://github.com/owenbrown/transcribe It's just a python script to test the streaming. Wow, Voxtral is amazing. It will be great when someone stitches this up so an LLM starts thinking, researching for you, before you actually finish talking. Like, create a conversation partner with sub 0.5 second latency. For example, you ask it a multi part questions and, as soon as you finish talking, it gives you the answer to the first part while it looks up the rest of the answer, then stitches it together so that there's no break. The 2-3 second latency of existing voice chatbots is a non-started for most humans.

jwblackwell 8mo ago
Nice! works well - I couldn't get huggingface to work either
jpeeler 8mo ago
Yes, appreciate this. I noticed that with both models voxtral-mini-transcribe-realtime-2602 and voxtral-mini-2602 filler words are ignored. I'd like to be able to count words/sounds, specifically "um" or "uh" for improvement purposes. Any good models that handle that?

barrell 8mo ago

Very happy with all the mistral work. I feel like I'm always one release behind theirs. Last time they released Mistral 3 I commented saying how excited I was to try it out [1] Well, I'm happy to report I integrated the new Mistral 3 and have been truly astounded by the results. I still am not a big fan of the model wrt factual information - it seems to be especially confident and especially wrong if left to it's own devices - but with http://phrasing.app http://phrasing.app I do most of the data aggregation myself and just use an LLM to format it. Mistral 3 was a drop-in replacement for 3x the quality (it was already very very good), 0% error rate for my use case (I had an issue for it occasionally going off the rails that was entirely solved), and sticks to my formatting guidelines perfectly (which even gpt-5-pro failed on). Plus it was somehow even cheaper. I'm using Scribe v2 at the moment for TTS, but I'm very excited now to try integrating Voxtral Transcribe. The language support is a little lacking for my use cases, but I can always fall back to Scribe and amatorize the cost across languages. I actually was due to work on the transcription of phrasing very soon so I guess look forward to my (hopefully) glowing review on their next hn launch! XD [1] https://news.ycombinator.com/item?id=46121889#46122612 https://news.ycombinator.com/item?id=46121889#46122612

qwertytyyuu 8mo ago

This exciting, especially after 11 labs very expensive model

ashu1461 8mo ago

Wondering if most of the AI agents use real time apis or transcription apis.. anyone had experience with building voice agents can comment ?

kranke155 8mo ago

Really cool.

RT_max 8mo ago

[dead]