toebee

5 ms·

1.
by toebee 11d ago
if we can lower the pricing by not 50% but 10x, then I think it would be something people want. we are taking the bet that OSS models will take a huge chunk of market share not just in LLMs but in multimodal as well
2.
by toebee 12d ago
thank you Dylan!
3.
by toebee 12d ago
agreed. we've been doing some work around NVIDIA personaplex 7b, but its quality is quite far from GPT-Live-1, esp in terms of intelligence. Once a good OSS model is out, we'll be sure to be the first to serve it cheaply to the ma
4.
by toebee 12d ago
awesome! will look into Darwin TTS. super interesting
5.
by toebee 12d ago
the qwen3-asr inference repo is not OSSed as of now. we're planning to write a paper or tech report on it as it contains some general techniques for ASR inference.
6.
by toebee 12d ago
hey, thanks for letting us know! will look into the issue and see what went wrong.
7.
by toebee 12d ago
very cool. will try for local use!
8.
by toebee 12d ago
thanks for the interest! we have a blog post on exactly how we did it: https://narilabs.com/blog/qwen3-tts-speed-cost-frontier/
9.
by toebee 12d ago
thanks for the feedback! will investigate and get it fixed
10.
by toebee 12d ago
hey sorry about that, you can here a few of our voices here: https://narilabs.com/product/qwen3-tts/
11.
Show HN: Nari Qwen3-TTS and Qwen3-ASR – High accuracy, low latency and cost(narilabs.com)
89 points by toebee 12d ago|31 comments
12.
Expanding the Pareto Frontier for Realtime Transcription(narilabs.com)
1 points by toebee 15d ago|1 comments
13.
by toebee 15d ago
Qwen3-ASR 1.7B is an excellent OSS Speech-to-Text model. But it is slow out of the box. We built an inference engine around it, and now it is the fastest realtime transcription model (as of Sep 2026) at 40 ms time-to-final-segment (TTFS). A
14.
Show HN: 10x cheaper TTS at 50ms time-to-first-audio(narilabs.com)
1 points by toebee 16d ago|1 comments
15.
by toebee 16d ago
Hey HN, Toby from Nari Labs here. Our TTS endpoints for Qwen3-TTS 1.7B are live: > "Fast" endpoint: client-side p90 time-to-first-audio (TTFA) at 50ms. > "Standard" endpoint: dirt cheap at $5 per 1M characters, sti
16.
by toebee 1mo ago
will look into this and get it fixed
17.
by toebee 1mo ago
will try to record a video soon, in the meantime you can grab a h100 from somewhere like runpod or modal and test it out: docker run --rm --gpus all \ -p 8000:8000 \ -e HF_TOKEN \ -e QWEN3_TTS_PROFILE=ttfa \ -v nari-qwen3-tts-cache:
18.
by toebee 1mo ago
will try to record something - in the meantime you can spin up a machine on runpod or modal to quickly test it out. docker run --rm --gpus all \ -p 8000:8000 \ -e HF_TOKEN \ -e QWEN3_TTS_PROFILE=ttfa \ -v nari-qwen3-tts-cache:/
19.
by toebee 1mo ago
we haven't tried so can't say for sure, but if optimized for a scenario where the batch size is 1 and max concurrency is 1, it seems possible to get something pretty fast. i'm guessing mobile hardware specific optimizations w
20.
by toebee 1mo ago
we continuously compared output to qwen's original implementation and do not see differences in output quality. let us know if it works well for you!
21.
by toebee 1mo ago
Yep, might need some changes.
22.
by toebee 1mo ago
We do graph capture etc at startup (same as vLLM) but this model variant doesn’t require prefix caching - the prefix is just 10 tokens.
23.
by toebee 1mo ago
Qwen3 TTS has input streaming mode: you can stream LLM output into the speech model. So don’t need to wait for a full sentence. We also implement this websocket variant, and it also runs at sub 50 ms. LLM TTFT is still a big issue, and we m
24.
by toebee 1mo ago
We got a rtx 4090 handling around 10 concurrent requests at 50 ms TTFA after some config changes / adjustment as it doesn’t have FP8. So this 50 ms TTFA thing is very much possible on consumer hardware.
25.
by toebee 1mo ago
Hey thank you for your kind words! Yes, we’ve shifted to inference but will also continue doing finetuning etc. on top of open models. Don’t have plans to do pretraining though.
26.
How we made a text-to-speech model respond in sub-50 ms(nari-labs.com)
171 points by toebee 1mo ago|44 comments
27.
by toebee 1mo ago
time-to-first-audio (TTFA) is critical for realtime voice applications. open source implementations (e.g. vLLM-Omni, SGLang-Omni) are often too slow for production and can have issues with realtime playback if you push for lower latency. we
28.
Show HN: Dia2, open-weights TTS model for realtime speech to speech(github.com)
3 points by toebee 10mo ago|2 comments
29.
by toebee 1y ago
You're absolutely right. We used Jordan's Whisper-D, and he was generous enough to offer some guidance along the way. It's also a valid criticism that we haven’t yet audited the dataset for existing list of tags. That’s somet
30.
by toebee 1y ago
not part of the license!

More ›