11 ms·
Show HN: Nari Qwen3-TTS and Qwen3-ASR – High accuracy, low latency and cost
Hey HN, Toby from Nari Labs here.
We've been working on making OSS speech models super-fast. Last year, we built Dia, the first OSS text-to-speech model capable of doing natural dialogue. Since then, so many more great speech models have been released to the public.
But the market is still dominated by closed source models. We think that's an inference problem. Existing systems such as vLLM / SGLang are not well suited for multimodal inference. To prove this, we built an inference engine specialized for Qwen3-TTS and open-sourced it (https://github.com/nari-labs/nari-qwen3-tts https://github.com/nari-labs/nari-qwen3-tts). Running at sub-50 ms latency at 10 RPS, this showed open models can be run much faster and cheaper.
Since then, we've been working hard to bring cheap, fast, and high quality serving to all. And we've even beat closed models at their game!
Measured on the highly cited Coval (YC S24) voice AI benchmarks, our Qwen3-TTS endpoint not just is #2 in latency, but #1 in accuracy (WER) compared to 11Labs, Cartesia etc. while being the cheapest endpoint. Our Qwen3-ASR endpoint has the lowest latency and #2 accuracy, just 0.1% away from #1. It is the second cheapest model on the list.
It took a lot of clever inference engineering to make these models quick, perform well while keeping costs low. Interestingly, Alibaba's official endpoints seem to perform worse in terms of accuracy and latency compared to ours. But nonetheless, much love to the Qwen team for OSS-ing these amazing speech models.
We want to continue to push prices down to make speech technology a commodity - so that every app can have great TTS and STT without worrying about unit costs. We're also working on other parts of audio such as diarization - as well as video and world model inference. More to come!
- ipsum2 13d agoIf you're going to announce a TTS model, service, or whatever, you really need demos.
- toebee 13d agohey sorry about that, you can here a few of our voices here: https://narilabs.com/product/qwen3-tts/ https://narilabs.com/product/qwen3-tts/
- emayljames 12d agoThe horizontal moving elements of examples become stuck and unable to be scolled once one of them is played. I'm using Vivaldi (chrome based) on Android
- rahimnathwani 13d agoFor some reason it switched voices half way through a 33 second clip. For OP the clip name is nari-nina-01a0a12f-980a-765e-8029-fa56bd23210d.wav
- toebee 13d agohey, thanks for letting us know! will look into the issue and see what went wrong.
- deleted 13d ago[deleted]
- asaiacai 13d agoThis is really cool work! I'm curious like what do you see as the biggest lever for speeding up TTS models or from a technical perspective that this was a promising direction in the first place to push on. If I were to guess, some distillation but I'm certain there are probably TTS model aware architectural changes that just make inference wayyyy faster?
- toebee 13d agothanks for the interest! we have a blog post on exactly how we did it: https://narilabs.com/blog/qwen3-tts-speed-cost-frontier/ https://narilabs.com/blog/qwen3-tts-speed-cost-frontier/
- meatmanek 13d ago> and Qwen3-ASR Is the ASR inference engine open source as well?
- nshm 13d agoYes, and it is very good one. Leading position on private leaderboard on HF: https://huggingface.co/spaces/hf-audio/open_asr_leaderboard https://huggingface.co/spaces/hf-audio/open_asr_leaderboard
- meatmanek 13d agoI meant the Nari inference engine for Qwen3-ASR. I'm aware that Qwen3-ASR is open source, but I don't see a repo under https://github.com/nari-labs https://github.com/nari-labs for nari-qwen3-asr or similar. The Huggingface link on https://narilabs.com/product/stt/ https://narilabs.com/product/stt/ links to https://huggingface.co/Qwen/Qwen3-ASR-1.7B https://huggingface.co/Qwen/Qwen3-ASR-1.7B , not anything under https://huggingface.co/nari-labs https://huggingface.co/nari-labs
- toebee 13d agothe qwen3-asr inference repo is not OSSed as of now. we're planning to write a paper or tech report on it as it contains some general techniques for ASR inference.
- kshmir 13d agohow do I follow you? I have a small 5090 doing inference all the time and I barely use tts but a lot of asr, mostly whisper, I ported your tech report for tts and implemented some improvements on my whisper inference based on your tech report as well! would love to talk sometime!
- verdverm 13d agoThey have a number of demos and examples in their HF space https://huggingface.co/Qwen/spaces https://huggingface.co/Qwen/spaces I saw a local-ai demo (something + gemma), where the person used ASR to get text and gemma to clean it up (like turning "question mark" into a literal "?", bullet points another one). The presenter also showed a gemma only option, that did both in one go, but had a higher WER on average, and even though the formatting statements were handled without a multi-stage pipeline, they preferred the multi-stage overall
- nthypes 13d ago[dead]
- mowmiatlas 13d agoCool, I’ve released something to the same beat of the dr this weekend as well https://github.com/loudreader/loudkit https://github.com/loudreader/loudkit I think real time natural tts should be possible everywhere soon
- toebee 13d agovery cool. will try for local use!
- yoloakki 13d agoYou definitely need independent evals by Datapoint AI or someone who can verify your claims about TTS quality
- DylanMerigaud 13d agoRooting for you on this one.
- toebee 12d agothank you Dylan!
- iharnoor 13d agoBy next month the competition for TTS will be even more! Voice models are not winner take all market unlike LLM APIs Coming here as Developer Relations at AssemblyAI
- konart 13d agoAll TTS generations are too fast. It's almost I'm listening to a podcast on 1.25-1.5x speed.
- toebee 13d agothanks for the feedback! will investigate and get it fixed
- bilaly 13d ago[flagged]
- apimade 13d agohttps://apimade.com/audio-compare.html https://apimade.com/audio-compare.html Added it to my blind TTS model comparison leaderboard. So far Darwin TTS is the open model leading the pack, ElevenLabs is at the lead.
- barney54 13d agoIs Darwin TTS from Fish Audio? It wasn't clear when I searched for it.
- apimade 13d agoDarwin TTS is based on Qwen3-TTS https://huggingface.co/zeropointnine/Darwin-TTS-1.7B-Cross-Qwen3Tokenizer https://huggingface.co/zeropointnine/Darwin-TTS-1.7B-Cross-Q...
- toebee 13d agoawesome! will look into Darwin TTS. super interesting
- karimf 13d agoThis is awesome. Thanks for pushing the audio pareto frontier forward. Probably far fetched for now, but I think the next big evolution is building the pareto/much cheaper alternative to GPT-Live-1. The STT/TTS market is quite saturated, while today, there's almost no cheap/open source alternative to GPT-Live-1.
- 6879346626 12d agoIs this really something people want? Honestly you can properly lower the pricing at least by 50%+. Getting something conversationally better has been done, the tool calling will likely be worse though. The infrastructure for real time is really annoying though.
- toebee 12d agoif we can lower the pricing by not 50% but 10x, then I think it would be something people want. we are taking the bet that OSS models will take a huge chunk of market share not just in LLMs but in multimodal as well
- 6879346626 12d agoIf audio only then 10x is 100% possible right now based on math of the services. + Video is unlikely unless they are willing to give up margins. Didn't take long at all for Google to smash out with 50% lol
- toebee 12d agoagreed. we've been doing some work around NVIDIA personaplex 7b, but its quality is quite far from GPT-Live-1, esp in terms of intelligence. Once a good OSS model is out, we'll be sure to be the first to serve it cheaply to the masses :)
- FirstClassTree 12d ago[dead]
- artexety 12d ago[dead]
- recentlypostedj 12d agoQuestion: How do you plan to differentiate, because there are so many TTS and its constantly changing every month who would become better