3 ms·

I got this running on macOS using mlx-audio thanks to Prince Canuma: https://x.com/Prince_Canuma/status/2014453857019904423 https://x.com/Prince_Canuma/status/2

by simonw 10mo ago

I got this running on macOS using mlx-audio thanks to Prince Canuma: https://x.com/Prince_Canuma/status/2014453857019904423 https://x.com/Prince_Canuma/status/2014453857019904423 Here's the script I'm using: https://github.com/simonw/tools/blob/main/python/q3_tts.py https://github.com/simonw/tools/blob/main/python/q3_tts.py You can try it with uv (downloads a 4.5GB model on first run) like this: uv run https://tools.simonwillison.net/python/q3_tts.py \ 'I am a pirate, give me your gold!' \ -i 'gruff voice' -o pirate.wav

gcr 10mo ago
This is wonderful, thank you. Another win for uv!
indigodaddy 10mo ago
Simon how do you think this would perform on CPU only? Lets say threadripper with 20G ram. (Voice cloning in particular)
- simonw 10mo ago
  No idea at all, but my guess is it would work but be a bit slow. You'd need to use a different build of the model though, I don't think MLX has a CPU implementation.
- genewitch 10mo ago
  the old voice cloning and/or TTS models were CPU only, and they weren't realtime, but no worse than 2:1, 30 seconds of audio would take 60 seconds to generate. roughly. in 2021 one-shot TTS/cloning using GPUs was getting there, and that was close enough to realtime; one could, if one was willing to deal with it, wire microphone audio to the model, and speak words, and the model would, in real time, modify the voice. Phil Hendrie is jealous. anyhow, with faster CPUs and optimizations, you won't be waiting too long. Also 20GB is overkill for an audio model. Only text - LLM - are huge and take infinite memory. SD/FLUX models are under 16GB of ram usage (uh, mine are, at least!), for instance.
genewitch 10mo ago
If i am ever in the same city as you, i'll buy you dinner. I poked around during my free time today trying to figure out how to run these models, and here is the estimable Simon Willison just presenting it on a platter. hopefully i can make this work on windows (or linux, i guess). thanks so much.
- cube00 10mo ago
  > hopefully i can make this work on windows (or linux, i guess). mlx-audio only works on Apple Silicon
  - bigyabai 10mo ago
    The original script supports CPU inference, nonetheless.
rahimnathwani 10mo ago
If you want to do custom voice cloning, record a sample wav file with a sentence or two, and then try this: uv tool install --force git+https://github.com/Blaizzy/mlx-audio.git --prerelease=allow python -m mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-0.6B-Base-bf16 --text "Hello, this is a test." --ref_audio path_to_audio.wav --ref_text "Transcript of the reference audio." --play

by simonw 10mo ago

gcr 10mo ago

This is wonderful, thank you. Another win for uv!

indigodaddy 10mo ago

Simon how do you think this would perform on CPU only? Lets say threadripper with 20G ram. (Voice cloning in particular)

simonw 10mo ago
No idea at all, but my guess is it would work but be a bit slow. You'd need to use a different build of the model though, I don't think MLX has a CPU implementation.
genewitch 10mo ago
the old voice cloning and/or TTS models were CPU only, and they weren't realtime, but no worse than 2:1, 30 seconds of audio would take 60 seconds to generate. roughly. in 2021 one-shot TTS/cloning using GPUs was getting there, and that was close enough to realtime; one could, if one was willing to deal with it, wire microphone audio to the model, and speak words, and the model would, in real time, modify the voice. Phil Hendrie is jealous. anyhow, with faster CPUs and optimizations, you won't be waiting too long. Also 20GB is overkill for an audio model. Only text - LLM - are huge and take infinite memory. SD/FLUX models are under 16GB of ram usage (uh, mine are, at least!), for instance.

genewitch 10mo ago

If i am ever in the same city as you, i'll buy you dinner. I poked around during my free time today trying to figure out how to run these models, and here is the estimable Simon Willison just presenting it on a platter. hopefully i can make this work on windows (or linux, i guess). thanks so much.

cube00 10mo ago
> hopefully i can make this work on windows (or linux, i guess). mlx-audio only works on Apple Silicon
- bigyabai 10mo ago
  The original script supports CPU inference, nonetheless.

rahimnathwani 10mo ago

If you want to do custom voice cloning, record a sample wav file with a sentence or two, and then try this: uv tool install --force git+https://github.com/Blaizzy/mlx-audio.git --prerelease=allow python -m mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-0.6B-Base-bf16 --text "Hello, this is a test." --ref_audio path_to_audio.wav --ref_text "Transcript of the reference audio." --play