3 ms·

Can anyone recommend an open source option that would allow training on a custom voice (my own, so I'd be able to record as many snippets as it needed to train

by swores 2y ago

Can anyone recommend an open source option that would allow training on a custom voice (my own, so I'd be able to record as many snippets as it needed to train on) to allow me to use it for TTS generation without sharing it off my machine? Edit: I'll wait to see if any recommendations get made here, if not I might give this one a go: https://github.com/coqui-ai/TTS https://github.com/coqui-ai/TTS

numpad0 2y ago
I think you can probably generate TTS audio by classical means, and voice2voice that audio through RVC or Beatrice V2. Haven't looked into it in a while but Beatrice is apparently super fast and CPU only.
phrotoma 2y ago
https://github.com/DrewThomasson/ebook2audiobook https://github.com/DrewThomasson/ebook2audiobook
esskay 2y ago
If I recall Coqui is very much a dead project, just one to be aware of.
hm64 2y ago
Coqui is great, but in practice, I found Piper easier to set up, train, and deploy as an ONNX file. Big thanks to the Sherpa development team for their helpful resources: https://k2-fsa.github.io/sherpa/onnx/tts/piper.html https://k2-fsa.github.io/sherpa/onnx/tts/piper.html and to the Rhasspy team for their training guide: https://github.com/rhasspy/piper/blob/master/TRAINING.md https://github.com/rhasspy/piper/blob/master/TRAINING.md. I also found DEMUCS + Whisper + pydub to be a super helpful combo for creating quality datasets.
drewbitt 2y ago
There is a fork here https://github.com/idiap/coqui-ai-TTS https://github.com/idiap/coqui-ai-TTS 'coqui-tts' Though according to the TTS leaderboard, Fish Speech https://github.com/fishaudio/fish-speech https://github.com/fishaudio/fish-speech and Kokoro are higher. https://huggingface.co/hexgrad/Kokoro-82M https://huggingface.co/hexgrad/Kokoro-82M https://huggingface.co/fishaudio/fish-speech-1.5 https://huggingface.co/fishaudio/fish-speech-1.5
- xnx 2y ago
  AFAIK Kokoro can't be fine tuned
jsemrau 2y ago
I wrote this a while ago about xTTSv2 mixed with Nvidia's Nemo. Maybe it kicks off your journey. https://jdsemrau.substack.com/p/teaching-your-agent-to-speak?utm_source=publication-search https://jdsemrau.substack.com/p/teaching-your-agent-to-speak...
eamag 2y ago
F5-TTS, I wrote a post about it https://eamag.me/2025/Voice-Cloning https://eamag.me/2025/Voice-Cloning

by swores 2y ago

numpad0 2y ago

I think you can probably generate TTS audio by classical means, and voice2voice that audio through RVC or Beatrice V2. Haven't looked into it in a while but Beatrice is apparently super fast and CPU only.

phrotoma 2y ago

https://github.com/DrewThomasson/ebook2audiobook https://github.com/DrewThomasson/ebook2audiobook

esskay 2y ago

If I recall Coqui is very much a dead project, just one to be aware of.

hm64 2y ago

Coqui is great, but in practice, I found Piper easier to set up, train, and deploy as an ONNX file. Big thanks to the Sherpa development team for their helpful resources: https://k2-fsa.github.io/sherpa/onnx/tts/piper.html https://k2-fsa.github.io/sherpa/onnx/tts/piper.html and to the Rhasspy team for their training guide: https://github.com/rhasspy/piper/blob/master/TRAINING.md https://github.com/rhasspy/piper/blob/master/TRAINING.md. I also found DEMUCS + Whisper + pydub to be a super helpful combo for creating quality datasets.

drewbitt 2y ago

There is a fork here https://github.com/idiap/coqui-ai-TTS https://github.com/idiap/coqui-ai-TTS 'coqui-tts' Though according to the TTS leaderboard, Fish Speech https://github.com/fishaudio/fish-speech https://github.com/fishaudio/fish-speech and Kokoro are higher. https://huggingface.co/hexgrad/Kokoro-82M https://huggingface.co/hexgrad/Kokoro-82M https://huggingface.co/fishaudio/fish-speech-1.5 https://huggingface.co/fishaudio/fish-speech-1.5

xnx 2y ago
AFAIK Kokoro can't be fine tuned

jsemrau 2y ago

I wrote this a while ago about xTTSv2 mixed with Nvidia's Nemo. Maybe it kicks off your journey. https://jdsemrau.substack.com/p/teaching-your-agent-to-speak?utm_source=publication-search https://jdsemrau.substack.com/p/teaching-your-agent-to-speak...

eamag 2y ago

F5-TTS, I wrote a post about it https://eamag.me/2025/Voice-Cloning https://eamag.me/2025/Voice-Cloning