3 ms·
The HF demo space was overloaded, but I got the demo working locally easily enough. The voice cloning of the 1.7B model captures the tone of the speaker very we
by magicalhippo 9mo ago
The HF demo space was overloaded, but I got the demo working locally easily enough. The voice cloning of the 1.7B model captures the tone of the speaker very well, but I found it failed at reproducing the variation in intonation, so it sounds like a monotonous reading of a boring text.
I presume this is due to using the base model, and not the one tuned for more expressiveness.
edit: Or more likely, the demo not exposing the expressiveness controls.
The 1.7B model was much better at ignoring slight background noise in the reference audio compared to the 0.6B model though. The 0.6B would inject some of that into the generated audio, whereas the 1.7B model would not.
Also, without FlashAttention it was dog slow on my 5090, running at 0.3X realtime with just 30% GPU usage. Though I guess that's to be expected. No significant difference in generation speed between the two models.
Overall though, I'm quite impressed. I haven't checked out all the recent TTS models, but a fair number, and this one is certainly one of the better ones in terms of voice cloning quality I've heard.
- thedangler 9mo agoHow did you do this locally? Tools? Language?
- magicalhippo 9mo agoI just followed the Quickstart[1] in the GitHub repo, refreshingly straight forward. Using the pip package worked fine, as did installing the editable version using the git repository. Just install the CUDA version of PyTorch[2] first. The HF demo is very similar to the GitHub demo, so easy to try out. pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128 pip install qwen3-tts qwen-tts-demo Qwen/Qwen3-TTS-12Hz-1.7B-Base --no-flash-attn --ip 127.0.0.1 --port 8000 That's for CUDA 12.8, change PyTorch install accordingly. Skipped FlashAttention since I'm on Windows and I haven't gotten FlashAttention 2 to work there yet (I found some precompiled FA3 files[3] but Qwen3-TTS isn't FA3 compatible yet). [1]: https://github.com/QwenLM/Qwen3-TTS?tab=readme-ov-file#quickstart https://github.com/QwenLM/Qwen3-TTS?tab=readme-ov-file#quick... [2]: https://pytorch.org/get-started/locally/ https://pytorch.org/get-started/locally/ [3]: https://windreamer.github.io/flash-attention3-wheels/ https://windreamer.github.io/flash-attention3-wheels/
- dur-randir 9mo agohttps://github.com/sdbds/flash-attention-for-windows/releases https://github.com/sdbds/flash-attention-for-windows/release... - FA2 binaries for you
- regularfry 9mo agoIt flat didn't work for me on mps. CUDA only until someone patches it.
- magicalhippo 9mo agoDemo ran fine, if very slowly, with CPU-only using "--device cpu" for me. It defaults to CUDA though. Try using mps I guess, I saw multiple references to code checking if device is not mps, so seems like it should be supported. If not, CPU.
- dsrtslnd23 9mo agoAny idea on the VRAM footprint for the 1.7B model? I guess it fits on consumer cards but I am wondering if it works on edge devices.
- magicalhippo 9mo agoThe demo uses 6GB dedicated VRAM on Windows, but keep in mind that it's without FlashAttention. I expect it would drop a bit if I got that working. Haven't looked into the demo to see if it could be optimized by moving certain bits to CPU for example.